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ABSTRACT 



A system and a method for monitoring a coinputcr network 
to detect data packets including audio or video data, such 
packets being part of a communication session, for storing 
these packets and for reconstructing the communication 
session upon request. 
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I 

COMMUNICATION MANAGEMENT 
SYSTEM FOR COMPUTER NETWORK- 
BASED TELEPHONIES 

FIELD AND BACKGROUND 

ITie present invention is of a method and a system for the. 
management ol' communication sessions for computer 
network-based telephone communication, and in particular 
for the idenlificalion of packets containing audio and/or 
video data, for the storage of these packets, and for the- 
y - /2-- / reconstruction of selected communication sessions for audio 
-and/or video display as needed. 

'ITie integration of the computer into office communica- 
tion systems has enabled many functions previously per- 
formed by separate devices to be combined into a single 
manage merit system operated through a computer. For 
example, computer-based voice logging systems enable a 
computer to receive voice communication through a hard- 
ware connection to the regular telephony network, to record 
either a conversation, in which at l east two parties converse , 
or a message from at least one party to one or more parties , 
and to replay these recorded conversations or messages upon 
request. These voice logging systems can replace mechani- 
cal telephone answering machines. 

computer logging systems have many advantages 
over the mechanical answering machines. For example, the 
voice messages can be stored in a computer-based storage 
medium, such as a DAT cassette, which has a greater storage 
capacity than regular audio cassettes. Furthermore, the 
stored voice messages can be organized in a database, such 
thai the messages can retrieved according to time, date, 
channel, dialed number or caller identification, for example. 
vSuch organization is not possible with a mechanical tele- 
phone answering machine. Thus, computer logging systems 
for voice messages have many advantages over mechanical 
answering machines. 

Unfortunately, currently available computer logging sys- 
tems have the tlisadvantage of being unable to record 
telephone communication sessions, whether conversations 
or messages, for voice communication being performed 
through a IAN (local area network) or a WAN (wide area 
network). Although these logging systems can play Ixick 
voice messages to a remote user through a LAN, for 
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SUMMARY OF IHE INVENTION 

It is one object of the present invention to provide a 
system and a method for recording communication sessions 
performed over a computer network. 

It is another object of the present invention to provide 
such a system and method for analyzing data transmitted 
over the computer network in order to detect audio and video 
data tbr recording. 

It is still another object of the present invention to provide 
such a system and method for displaying recorded video and 
audio data upon request. 

It is yet another object of the present invention to provide 
such a system and method for analyzing, recording and 
displaying communication sessions conducted with a LAN- 
based telephone system. 

These and other objects of the present invention are 
explained in further detail with regard to the drawings, 
description and claims provided below. 

The present invention provides a system and a method for 
analyzing data packets on a computer network, for selec- 
tively recording audio and video data packets, for organizing ' 
this stored information and for displaying the stored infor- 
mation upon request, such that communication sessions with 
computer net work -based ''telephone'' systems can be 
logged . 

According to the teachings of the present invention, there 
is provided a system for managing a communication session 
over a computer network, the system comprising: (a) a 
network connector for connecting to the cx>mputer network 
and for receiving data packets from the computer network; 
(b) a lUleriru^nit for lijiering the da ta pac kets and for-^ 
accepting the data packets substantially only if the data / 
packets cont;)in data selected from the group consisting of f 
a udio-dat a and video data, such that the data packets form at ( 
least a portion of the conununication session and such that \ 
the data packets are selected data packets; (c) a managements^ 
unit for r eceivin g the select ed dat a packe ts and for storing 
the selected data packets, such that the s elected dat a packets 
are s tored data packe ts; and (d) a storage medium for 
receiving and for storing the stored data packets from the 
management unit, such that the at least a portion of the 
communication session is stored. 

Preferably, the system further comprises (e) a d ata resto re ' 



example, they cannot record such a message if it is trans- 45 unit for r etrieving a nd d isplay in g the at least a portion of the 



mitted by a LAN-based lelcphone. Such LAN and WAN 
l);ised tolcpfi6iic conununication has become more jjopiilar 
n;rcntly, since i! enables telephone communication to be 
performed between va rious partie s al physically separated 
sites witlunii paying for local regular telephony network 
services, thereby saving money. 

I'lirilierniore, LAN and WAN based telephone communi- 
cation also facilitates the transmission of videt> as well as 
audio information. Video information certainly cannot be 



communication session, the data restore unii requesting the 
data packets frotn llieslor;ige fiicdiiim llimugli llic nian;ige- 
ment unit, and the data restoxe^un it rccousf ^ uctin^^ the da ta 
n ackets-fur-disnlav inii the al least a portion of the commu- 
nication session. 

More preferably, the data restore imit titrl her comprises a 
comoiunication session display unit for displaying the at 
lens; a portion o\' the communication session. Most 
preferably, the communication S'j^sion djjjpiay unit is 



recorded by currently available computer logging systems. 55 s elect ed from the group consisting of a yklet) unit and an 



llius, tile inability of computer logging systems to record 
telephone comnumicalion sessions for telephone communi- 
cation being performed through a LAN or a WAN, including 
both video and audio data, is a significant tlisadvanlage of 
these systems. 

'There is therefore a need for, and it would be highly 
advantageous to have, a system and a method for ivcording 
telephone communication sessions performed over :i com- 
puter network such as a l,AN or a WAN, which would record 
lioth audio and video information, organize such 
information, and I lien display such infornuition upon 
request. 



a udio_Jiin it. 

According to preferred embodiments of the present 
invention, the system further cximprises (I) a database con- 
nected to the filtering unit for s toring liltcrinu inforniajio n, 
the li ]toring information including at least o ne lljaddre ss of 

party vvKose^ommunicniion sessions are n] paitia jxd; 
wherein the tiltering unit accepts the data packets according 
to Itie fillering informal ion, such that the (iltering unit 
substantially only accepts the data packets if the data packets 
fulfil I the filtering information. 

Pretcrably, the system further comprises (g) ;i user com- 
puter for r eceivini^ at least one_ command of a user and for 
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displaying information to the user, such that the user deter- 
mines the filtering information according lo the at least one 
command of the user. 

More preferably, the computer network is selected from 
Ihe group coasisting of a LAN (local area network) and a 
WAN (wide area network). Most preferably, the computer 
network is a LAN (local area network). 

According to further preferred embodiments of the 
present invention, the LAN is divided into at least two 
segments, the system further comprising: (h) a local man- 
agement unit for each segment, the local management unit 
including the filtering unit and the management unit; and (i) 
a central nianagetnenl unit for controlling the local manage- 
ment units, the central management unit controlling storage 
in the storage medium. 

Preferably, the network connector is a network interface 
card. 

According to another embodiment of the present 
invention, there is provided a method tor storing at least a 
portion of a communication session performed on a com- 
puter network, Ihe communication session being performed 
between a packet source and a packet destination, the steps 
of the method being performed by a data processor, the 
— method comprising the steps of: (a) receiving a data packet 
frcMii the packet source on the computer network; (b) ana - 
l yzint; the data oackej to d etermine if the data packet is an 
IP packet; (c) if the data packet is the IP packet, f UterintJ th e 
I P packet to determine a type of the IP packe t ; and (d) stt^rin u 
the IP packet to form a stored data packet ac cording toj he 
l yge , such that the stored data packet forms at least a portion 
of the communication session. Preferably, the step of ana- 
lyzing the data packet is performed by e xamining a head er 
^ o 1 : the data pac ke t . 

According to a preferred embodiment of the present 
invention, the step of filtering the IP packet is performed by 
examining the header of the IP packet. 

Preferably, the step of filtering the IP packet further 
comprises the steps of: (i) examining the header of the IP 
packet to determine an IP address of the packet source; (ii) 
determining if the IP address is a recorded IP address; (iii) 
passing the IP packet to form a passed IP packet substan- 
tially only if the IP address is the recorded IP address; and 
(iv) alternatively, dumping the IP packet. 

More preferably, the step of determining if the IP address 
is the recorded IP address is performed by comparing tho IP 
address (o a list of IP addresses from packet sources, .such 
thai if the IP address is included in the list, the IP address is 
the recorded IP address. 

Also preferably, the step of Idlering Ihe IP packet further 
comprises the steps ot": (v) determininu w hether the pa.ssed 
IP p aclvei is an 11.22.^ packe t, a 1 1.24.^ packe t, an K'I'P packe t 
or an K'l'CP packe t; (vi) if the l yi)e ( )r the passed IP packe t 
is the 11.225 packet, determining v/helher the 11.225 packet 
is a s etup packe t or a c onnect packe t; (vii) if the 11.225 
packet is the s etup p; fcket. setting a status Hag as * \surt_ 
s essionj::eiJiie st": (viii) alternatively, if the 11.225 packet is 
the c onnect' pack et and Ihe s tatus Hail is '^st art sessio n 
request", storing at least one detail of the communication 
session; and (ix) s ettin g Ihe st atus ll air as " wait for logic 
chagiiej ". 

More preferably, the .step of filtering the IP packet further 
compri.ses the .steps of: (x) alternatively, if the type of the 
passed IP packet is the 11.245 packet, determining whether 
the 11.245 packet is an open logical channel request packet, 
an open logical channel acknowleclgrnent packel or a ter- 
minal capability set packet; (xi) if the 11.245 packet is the 
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open logical channel request packel and the status flag is 
"wait for logic channel", setting the status flag as "wait for 
acknowledgment"; (xit) alternatively, if the H.245 packet is 
the open logical channel acknowledgment packet and the 
status flag is '*wait for acknowledgment'', performing the 
steps of: (A) setting the status flag as "wait for terminal 
capability"; and (B) saving a transport address of the des- 
tination of the communication session; and (xiii) also 
alternatively, if the 11.245 packet is the terminal capability 
set packet, performing the steps of: (A) storing a capability 
of the packet destination from the terminal capability packet; 
and (B) setting the status flag as "in call process*'. 

Most preferably, if the sJaluaJlag is "in call process " and 
the l^e of the passed IP packet is the RTP packe t, the RTP 
packet is stored. Also most preferably, if the status flag is "in 
call process" and the t ype of the passed | p pfirkt^ i is the 
U rCP pack £l, the RTCP packet is stored. 

According to another preferred embodiment of the present 
invention, the method further comprises the steps of: (e) 
retrieving the stored data packet to form a retrieved data 
packet; and (i) reconstmcting at least a portion of the 
communication session according to the retrieved data 
packet. 

Preferably, the step of retrieving the data packet includes 
the steps of: (i) receiving a source IP address of the packet 
.source, a start time of the communication .session, and an 
end time of the communication session; and (ii) selecting at 
least one communication session according to the source IP 
address, the start lime and the end time. 

Also preferably, the step of reconstructing at least a 
portion of the communication session includes displaying 
audio data . 

Alternalively and also preferably, Ihe step of reconstnjcl- 
ingat least a portion of the communication session includes 
displaying video data. 

More preferably, the step of recoastructing at least i\~ 
portion of the communication session further comprises the 
steps of: (i) retrieving substantially only RTP packet.s; (ii) 
examining a header of the WW packets to determine a lime 
stamp for each of the RTP packets; and (iii) displaying the 
K\V packets in an order according to the time stamp. 

Hereinafter, the term "communication session" includes I 
both a conversation, in which at leasj two parties convers e 
by e.\changing audio and/or video information in "reid 
l ime;^ , and a mess^ige, m which at least o n ,t^ party r c^c<^rd s 
s uch and i <:> and/or video information for reci^j^t ion by ^ aj Jeast 
o ne other parly al a la t er dat e . 

Hereinafter, the term "Internet" is used to generally des- 
ignate Ihe global, linked web ol' thousands of networks 
which is u.sed lo connect computers all over Ihe world. As 
u.sed heroin, the lenn "intranet'' includes other types of 
computer niil works, such as LAN (local atea networks) or 
WAN (wide area networks). The term "c omputer networ k" 
inclu des any connection between at leasi two computer s 
w'liieh pc rnt 1 (-' ^^[he^ t ra rism ission g f_cLalji ^^,,^i nc luci ing _ both 
liilernet aiid iiijranet. The term "regular telephony network" 
includes PO TSTplain old telephone system) anti substan- 
tially any other type of telephone network which provides 
services through a regular telephone .services provider, l"»ut 
which specifically excludes audio and/or video comnnuii- 
caiion performed through any type of computer network. 

Ilereinafier, the term " compu ter'' includes, Init is not 
limited to, per.sonaI computers (PC) having an operating 
system such as DOS, Windows'^", OS/2"^" or Linux; Mack- 
inlosh^" computers; computers having .IAVA'''"-OS as the 
operating system; and graphical workstations such as the 
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computers of Sun Microsysiems'^" and Silicon Graphics'™, (LAN) which do not provide a guaranteed quality of service, 

and other computers having some version of the UNIX Computer terminals and equipment which fulfill H.323 may 

operating system such as AIX or SOLARIS''" of Sun carry real-time voice, data and video, or any combination. 

Microsystems™; or any other known and available operat- including vidcotelcphony. 

ing system. Hereinafter, the term "Windows''"" includes but 5 The LAN over which such terminals communicate can be 

is not limited to WindowsgS™. Windows 3.x''^ in which "x" a single segment or ring, or optionally can include muUiple 

I ^ > I Is an integer such as ''V\ Windows NT''". Windows98*'", segments with complex topologies, 'fhcse teiTninals are 

Windows CE™ and any upgraded versions of these oper- optionally integrated mlo compmers or alternatively are 

ating systems by Microsoft Inc. (Seattle, Wash., USA). implemented in stand-alone devices .such as vidcmde- 

II ■ .u * ui ■ ^ t «f in phones. Support for v oice data is required, while suooort for 

Hercmafter, the term logging reters to the process ol , , / . , , . \ \ . 

, . ' 1 , * 1 * I * r 1/ general data and v_ide o data arc optional, but ir supported, the 

analyzmg data packets on a network to locate audio and/or fs*,T.^v — t — » . 

• t , r r I. 1 I • ..•^^,1 ability to use a so ecined commnn mode or operatio n is 

video data, and ol recording such data in an organized / . . ^ . \ ^ r--*^ 7 , — 3^ 

. J, , I « • 1 I u *i. »u required, so that all terminals supporting th at pa rticular «^ 

A x^ystera. Hereinafter, the term display includes both the.^ r ' • . ^ru ii — — TT- 

y^^s^w^ Ci , , C I 1 . 1*1. J *• c tr media ty pe can communicate. Ihe H.323 Recommendation tsy 

# -> tr J>visuadisp ay of video data, and the production of sound for- , , ..•—TV ... M 

^-^ ^audio data 15 allows more than one channel ot_eachJ^j)e to be in us e, j v 

Other Recommendations irTthe H.323-Ser ies which are also I vlor^ 

I^RIEF DESCRIPTION 01^ THE DRAWINGS incorporated by reference include H.225.0 packet and syn- ^ CM'''^*^ 

chronization; H.245 control, H.261 and H.263 video codec s; ,<^y 

'ITic invention is herein described, by way of example G.711, G.722, G.728, G.729, and 0.723 audio codec s; and 
only, with reference to the accompanying drawings, ^^^thc T.120-Series of muhimcdia communications protocols. J ^ 

ITU-T Recommendation H,245.0 covers the definition of 

FIG. 1 is a .schematic block diagram of an exemplary Media stream packctization and synchronization lor visual 

communication .session monitoring system according to the telephone .systems. FIU-T Recommendation H.245.0 defines 

present invention; the Control protocol for muUimedia communications, and is 

FIG. 2 is a .sclicmatic block diagram of the software 25 hereinafter referred to as "H.245'". H. 24.5 is incorporated l>y 

modules required for operating the system of FIG. 1; reference as is fully set forth herein. 

FIGS. 3 A-3D are llowcharts of exemplary filtering and The logic al channe l signaling procedures of 11.245 

recording methods according to the present invention; describes the content' of e ach logical channe l when th e 

FIGS. 4A-4I:) arc .schematic l>lock diagrams showing the channel is opened . Procedures are provided tor the commu- '^^k^jl^ 

headers of H-225 (FIG. 4A), H.245 (FIG. 4B), RTF (FIG. -^^ nicalion oT the li inctional capabilities of receivers and 

4C) and RTCF (F'IG. 4D) packets, as they relate to the tra nsmitters, so that transmtssions are limited to intbrmati on 

p rcsc n t in ve n t ion ; ^^'^ ich ca n be decOdeclby the receivers , and so t ha 1 r eceiver s 

FIG. 5 is a llovvchart of an exemplary communication may re^t a particular desired mode from t ransmitter s, 
session playback method according to the present invention; " 245 signaling is established between two endpoints: an 

^. . . Ill ,1 „ ^r„„ r.,^. endpoint and a rnylti-point controller, or an endpoint and a 

hIG. 6 IS a schematic block diagram ot an exemplary tirst ^ » ^, ^ — f—*, rr^ , r.^^r ./ - 1 

... . r 1 • , . Gatekeeper. The endpoini establtshcs exactly one H.245 AAUC-fipte 

embodiment ol: a ba.sic system using the communication , * l l j • • • • *fr ^* 

• .«^.i-i7ir'o 1 -7 « , Control Channel lor each cal l that the endnomt is uartic i- 

session monitorinii sy.stem ol FIGS. 1 and 2 according to the * 1 j ■ — . -p=r=5f™=sr- , — ^ ^ — — ^ 

^ , paJUOS^in. The channel must then operate according to 

present invention; and f^Jc o * r u- t n 11 *• i7- 1 

' „ . . ^. , 11.245. Support lor muluak_calls and hcncx lor mul tiple. 

MG 7 IS a schematic block diagram ol an exemplary ,,^45 ^^^^^^^j Chann^^TSTe. 
.second embodiment of a zone system according to the . T inncri 

resent invention si gnaling function uses l l.22.'>.0 mes.sages to 

^ perform registration, admissions, bandwidth changes, status, 

DESCRIP'nON OF I^ACKGROUND ART and d isen^^c procedures between end | j) oints and Gatekeep- 

.... , ers. in LAN environments that do not have a Gatekeeper, the 

The tollowing description is intended to provide a 43 ^aS Signalinu Channel is not u.sed. In 1..AN environments 

description ofcertain background methods and technologies ^^^.^^ contaiif a Gatekeeper, such that the LAN includes at 
which arc (^Mionaliy u.sed m the method and .system ol the . y^^^^^^ Signaling Channel is <^4iiaia| 

pre.scnl i.iveinion. The present invention is .specihcally not endpoini uud the Gatekeeper. The RAS Signal- 

V drawn to these methods and technologies alone Rather, they • ^^^-.^^ establishm ent of any other 

are used as tools to accomplish the goal ol the present 5,, diannets betweei 11.323 endpoints: 
V invent ion, which is a system and a method lor analyzing data ^.^^^^.^^^^^ ^^^^ ,25.0 call signaling to 

■ (C\ / ^packets on a computer network, lor selectively recording . . r t ?• \ . . ni-t-^ 1 • , ^'i 

DO/ / . . . I . r ■ ■ .1 ■ . 1 esiahiish a connection between two 11.323 endpoints. Ihe 

' / / audio and video data packets, hjr organizinu this stored ., „. • 1 t . i- <i »acwm 

/ . . t • " (.all Siiiiialini; (.hannel IS iiiuenoiKlenl ln)in the KAS C.li;iii- 

/ tnlormation and lor displaying the .stored inlornuition upon , T . ti n^r . i At 1 -i-i rn r 

. . . * ncl ano the 11.245 Conltol Channel. Ihe Call Signaling 

I request, such that communication sessions with computer , . 1 • * .u » 1 r u . i- .1 no^c 

V , 1 1 II 1 Channel is opened prior to the establLshment or the 11.245 

^ network-ba.sed telephone .systems can be logged. ... , j 1 • 1 u t 1 . 11011 

* Channel and any other logical channels between 11.323 

The system and method ol: the present invention is par- endpoints. In .systems that do not have a Gatekeeper, the Call 

ticularly intended lor operation with computer networks signaling Chimnel is opened between the two endpoints 

constructed according to the ITU-T Recommendation 1 1.323 • -^^ j,^^ ^..^^ ^y^(^^,^ Gatekeeper, 

for visual lelej.lv.rw'. .systems and equipment lor local area 00 ^^^.^^^ Signaling Channel is opened l^etween the end pcini 
networks which provide a non-guaranteed quality ol .service j,^^ c;,jiekeep^er, or between the endpoints tliem.sclvcs as 

Recommendation 11.323 is herein incorporated by reterencc chosen by tiic Gatekeeper 
in order to fnriher describe the hardware re(|tiirenients and 

operating protocols for such computer nelwork.s, and is OFSCRll'I'lON OF TUB FRBI^'ORRBD 

hereinafter referred to as "11.323". (ks LIMBODIMBNTS 

11.323 describes lermiiials, ei]nipinent and services for The present inveni ion provides a system and a met hoti far 

multimedia communication over Local Area Networks analyzing data packets on a computer network, for .selec- 
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lively recording audio and video data packets, for organizing 
this stored informaiion and for displaying the stored infor- 
mation upon request, such that communication sessions with 
computer network-based ^'telephone" systems can be 
logged, 

'ITie principles and operation of a methoci and a system 
according to the present invention may be better understood 
with reference to the drawings and the accompanying 
(lescriplion. 

Referring now to the drawings, FIG. 1 is a block diagram 
of an exemplary system for logging and displaying audio 
and/or visual data from communication sessions performed 
over a computer network. A computer logging system 10 
features a user computer 12 connected to a communication 
session management unit 13. Communication session man- 
agement unit 13 is in turn connected to an intranet 14 
through a network interface card (NIC) 16. 

User computer 12 includes a u.ser interface 18, which is 
preferably a GUI, (graphical user interface), which Ls dis- 
played on a display unit 20. User interface 18 preferably 
enables the user to enter such information as the definition 
of the parties wiiose calls. should to be monitored and/or 
logged, and which also preferably enables the user to enter 
aTTeast one command for retrieving and displaying a com- 
-munication session. 

Display unit 20 is preferably a computer monitor. The 
user is able to interact with user computer 12 by entering 
data and commands through a data entry device 22. Data 
entry device 22 preferably includes at least a keyboard or a 
pointing tievice .such as a mouse, and more preferably 
inc hides both a keylioard and a pointing device. According 
to one preferred embodimeni of the present invention, u.scr 
computer 12 is a PC (personal computer). Alternatively and 
Ipreferably, user computer 12 is a "thin client" .such a net 
computer which is a computer able to communicate on an 
IP-based network. One example of such a net computer is 
the JavaStation'^''^^ (vSun Microsystems). The advantage ol 
such net computers is that they allow the user to interact with 
complex, sophisticated software programs, yet generally do 
not have all of the powerful computing capabilities of 
currently available PC computers. 

Intranet 14 could be a LAN or a WAN, for example. The 
connection l^eiween communication .session management 
imil 13 and intranet 14 occurs through NIC 16. NIC 16 is 
preferably any standard, olf-1 he-shelf commercial product 
which enables communication session management unit 13 
to !:e COM licet e( I to any .suitable computer network (for 
example, Lilherlink II KSA/PCMCIA Adapter or l-lherlink III 
PCI Rus-Masler Adapter (3c590) of 3-Com'f^\ or NI220()(I 
Adapter of Novell™ or any other such suitable product). 
Examples of such suitable computer networks include, bul 
are not lintitetl to, any standard I AN such as tithernel (llTil ! 
Standard 802.3), l^'asl tithcrnet (ItiBB Standard X(»2.l(»), 
Token King (IliliB Standard 802..'>) and f'DDI. 

All data packet tgdlic on rnlranet^UJ .'iQas.setLtCLa Jittering 
m odule 24 through NIC 16 . Asstiown in more detail in 1*10. 
3 below, filtering module 24 screens the data packets in 
order to determine which data packets fulfil I the following 
criteria. Rrietly, »!ic data packets should be IP packets with 
headers according to the 11.225 and H. 245 standards, indi- 
catin t^ voice and/or video traHiC . As noted previously, i lie.se 
slarulattls (lelirie nietlia .stream [lackcl construction and syn- 
chronization for visual telephone sy.stems and the conhol 
protocol for multimedia communications. 

fMllering modiiliy^4/hen preferably pas.ses substani ially 
oiily those data packe ts which meet these criteria to a 
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management module 2S. In the Zone Configuration of the 
system of the present invention, shown in FIG. 7 below, 
filtering module 24 preferably also transfers messages from 
other communication session management units. 
5 Management module 28 re ceives the data packets passed 
through by filtering module^ 4. and analyzes the received 
d ata packe ts. Optionally and preferably, a database 26 stores 
such information as the IP addresses of parties whose 
communication sessioas should be logged, as well as the 

10 conversion table associating each party with at least one IP 
acklress, for example. 'ITic stored list of IP addresses repre- 
senting those parlievS whose calls should be logged is pref- 
erably user-defined. As used herein, the term "party" refers 
to a person or persons communicating through a computer 

'5 network-based telephone system. ITie latter preferred 
requirement significantly reduces the amount of data stored 
by including pnly data which is of interest to the user^ 
Manag ement module 28 anal vzes_and_manages data in 
accordance with the applicable 1 1.225 and IJ .245 

'^^ spe cificatiQ Qs. including the H.245 control functio n, RAS 
sig naling func tion and c all signaling funct ion, substantially 
as described above in the 'Description of the Background 
Art" section. *• 
Manaj^ement module 28 an alyzes the packet s in order to 
d etermine the spec[fi c com munication session to which the 
d ata packets be I o n a. the type of da ta co mpression bei ng use d 
(if any), and wh ether the data packet s were .sent from an IP 
address which should be monitored . Management module 
28 must perform t his analysi^s since filtering modul e 24 
simply p asses all data packets which liillill the crite ria 
described brieily above (see I'lGS. 3A-3D for more detail). 
Since these packets are passed without regard to any of the 
information stored in database 26, management mcxlule 28 
mu.st compare the rules of database 26 (o the information 
present in the packet header of each packet in order to 
determine whether the received packet should be stored. 

Those received packets which fulfill the rules of database 
26 are then stored in a storage medium 30, which is 

^^-j preferably a high cjipacity digital data storage device such as 
a hard disk magnetic storage device, an optical disk, a 
CD-ROM, a ZIP or DVD drive, or a DAT ca.ssette, or a 
combination of such devices according to the operational 
needs of specific applications, or any other suitable storage 
media. Preferably, Ihe .specific communication session or" 
"telephone call'', with which each data paeket is associated, 
is also .stored in order for that .session to be reconstructed and 
displayed at a later lime. 

Upon request by tlie u.ser, management m odule 2 8 can 

5j) I hen r etriev e one or more data pack ets Iroin .storage medium 
30 which are a.ssocialed with one or more communication 
sessions. The r eUiey ed packet or packets are then tnni?yjiiijxil 
[o a da I a restore module^Sz ^3aia restore module 32 is 
pre fe7a51y capable of manipulating these retrieved packets 

55 to restore a particular c(uiimuiiicalio u_sessimi by using the 
l^fP ( Rcj' 1 - I i "I'J. ^Pr.otocaO.^As described in further detail 
below with regard to MGS. 4C and 5, in those sy.stems 
which follow the R TP, the data packets are sent with a lime 
stamp in the header rather than ju.st a sequence number. Such 
a lime stamp is necessary for audio and video .stream dai.i 
in order for the data packets lo be reassembled such that the 
overall timing of the stream of data is maintained. Without 
such a time stamp, Ihe proper liming would not be 
maintained, and (he audio or video .streams could not be 
accurately reconstructed. 

The connnunicalion sessions are resloretl from the recoH- 
stmcted streams of data packets by using the applicable 
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audio ancl/or video CODECs. A CODEC is a non-linear 
method tor the conversion of analog and digital data. Thus, 
an audio COD IZC enables the digitized audio data in relevant 
data packe ts to be converted to analog audio data fo r displa y 
t o the uscf as aucl ihir. ^nnnHs, for example. Suitable 5 
CODE C'S are described in greater detail below with regard 
to FIG. 5. 

In order Tor the user to receive the display of the recon- 
structed communication session, system 10 preferably fea- 
tures an audio unitCS ^arui a video unit 36, aiUectively 10 
referred to as a "communication session display unit". More 
preferably, both audio unit 34 and video unit 36 are capable 
of both receiving audio or video input, respectively, and of 
displaying audio or video output. At the very least, audio 
unit 34 and video unit 36 should be able to display audio or 
video output, respectively. For example, a udio unit 34 cou ld 
o ptionallv incl ud^u n microp hone for input and a speaker or 
an earphone for outp ut. Video unit 36 could optionally - 
include a video monitor or display screen for output and a - 
video camera for input, for example. ^'-'^ 

FIG. 2 is a schematic block diagram of system 10 of FIG. 

I, showing the overall system of software modules of system 
10 in more detail. Reference is also made, where 
appropriate, to How charts showing the operation of these 
software modules in more detail ( FIGS. 3A-3D and FIG. 5), 
as well as to dascriptions of the headers of the dilfcrcnt types 
of data packets (FIGS. 4A-4D). 

As shown, .system 10 again includes a connection to 
intranet 14 through NIC 16. As the packets are transmitted 
through intranet 14, NIC 16 intercepts these data packets and 
passes them to liltering iriodule 24. 

Filtering module 24 has two components. A f irst filter ing 
eomponent^ffi xamines the header of the data packet, which 
should be an I P type packe t with the correct header, as 
shown ill FIG. 4A below. Next, first filtering component 38 
passes the d ata pa cket to a second filter m a component 40. 
Second filtering componenty^lT^en determines th e type of 
I P data pack et, which could neconslructed according to the 

II. 225, II.24.5, RIP or inCP standards. 
As shown with reference to FIG. 3 A, fi rst filter in a co rn- 

p onent_3 8 and second filtering component^^p operate as 
follows. In step one, a packet is received by fil tering modul e 
24. The packet is given to fmst filtering c omponent j.S . which 
tlien determines w hether the packet is ;m IP [ y pc packet in 
.step two. Such a determination is performed according to the 
str ucture of tin; heade r of the data packet, an example of 
whit:ii is shown :ii i*'lG. 4 A. A header 42 is shown iis a^ — 
plurahty of boxes, each of which represents a portion or 
"field" of the header. The number of bytes occupied by each so 
portion is also shown, U l)eiiig miderstood that each layer 
consists or32 ImIs. The fust portion of the header, a VflRS" 
portion 44. is the protocol version numbei'. Next, an "I I. 
I..EN" portion 46 indicatus the number of 32-hit quantities in 
the header. A "SERVICE ^fYPE' portion 4K indicates 
whether the sender prefers the datagram to travel oy.cr a 
r qut_e with minimal delay or a route with maximaLthro.imh - 
pu^A " rO'i'AL I.ENG TIT' portion 50 indicates the total 
numfier of octets in both the header and the data. 

In the next layer, an ^M J jEN^TlFICATION" :;ortio. n 52 
ide ntifies the packet itself. A "l^'LAGS" portion 54 indicates 
whether the data g ^ ram is _a fr a gment or a complete datagram. 
A "iniAGMENrnTPr^^ 56 species the k)calion 

of this fragmeni in the original datagram, if the datagram is 
fragmented. In the next layer, a "TIME TO LIVE" portion ts^ 
58 contains a positive integer between I and 255, wliicli is 
progressively decremented at each route traveled. When the 
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value becomes 0, the packet will no longer be p assed and is 
returned to the sender. A '^"rvpp" pnnmT^Swnfi.ViiiPc the 
typ e of data bcin^ passed. A "HEADER CHECKSUM" 
portion 62 enables the integrity of the packet to be checked 
by comparing the actual checksum to the value recorded in 
portion 62. 

The next layer of header 42 contains the source IP address 
64, after which the following layer contains the destination 
IP address 66. An optional IP OmONS portion 6« is 
present, after which there is padding (if necessary) and a 
data portion 70 of the packet containing the data begins. 

The structure of the header of the data packet is examined 
by first filtering component 38 to determine whether this 
header has the necessary data fields in the correct order, such 
that the header of the data packet has a structure according 
to header 42. First filtering component 38 only allows those 
packets with the correct header slmcture to pass , as shown 
in step 3 A. Otherwise, the packets are dumped as shown in 
step 3B. 

Those packets with the correct header, or "IP packets", are 
then pas.sed to ^'f^opd tiliiTin^ fomp^gpni 40 Second lil- 
tering component 40 then performs the remainder of the 
filtering steps. In step 3 A, .second liltering component 40 
e xamines the IP packets to determine th|ei^ ^[Y[;>c from the 
d ata portion <)f the packcLiis^shown in I'lG. 4A. ITie packets 
could be in one of tpur c ategories: 11 .225, 11.245, R'F P and 
R TCP. -rhe steps of the method for 1 1.225 pac kets arc shown 
in FIG. 3A, while the procedures for the remaining packet 
types are shown in FIGS. 3B-3D, res[x;ctively. 

Once the type of the packet has been determinetl, both the 
packet, i tself and the informati on r ega r ding the type o f 
packet are both jja^d to management module 28, as shown 
in FIG. 2. ITie packet is then passed t o the relevant com - 
po nent within management module 28,ials o as shown in 
I'lCi. 2, for the recording process to be perforrried. The 
recorded packets are stored in storage module 30. as 
described in greater detail below with regard to FIGS. 3C 
and 3D. 

If the packet has been determined to be an 11.225 p acket 
according to the header of the packet (see FIG. 4M), the 
packet is passed to an 11.225 call ^ c o ptLaLmoil^ 78 within 
management module 28, as shown in FIG. 2. 'Hie steps of the 
management method are as follows, with reference to I'lG. 
3A. In .ste|) 4Aof h\G. 3A, the 11.225 packet is e.vainined to 
see if it is a setup packet, which is determined ;)ccor<ling lo 
the structure of the data in the packet. This structure is 
specified in tlie Il.225.(> recommcntlation, arvl includes ai 
least the following types of information: 

p rojioco 1 1 de n 1 i li er^ (the version of 11.225.0 which is 
sn(ip(Hled); 

h245 Address (speciuC transport adtlrcss on which 11.245 
signaling is to be established by the calling endpoini or 
gatekeeper); 

sojirce Address (the H.323_JD's for the source); 
>;o^|rc-rlnfo (contains an End point I ype to enable the party 

being called to determine whether the call includes a 

gateway or not); and 

de.stinatioiiaddrej»s (this is the address to which the end- 
point wants to be connected). 
Other types of data are also required, as specified in the 
Il.225.(» Recommendation. This data structure enables 
11.225 call control module 78 to d etermine whether the 
packeJLis_ a seUi p_|iacket . 

If this packet is a setup packej , then the first l)ranch of the 
method is followed. 'Flie st)urce port is taken from a source 
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port field 74 of an H.225 header 72, and the destination port 
is taken from a destination port field 76 (see FIG. 4B). In 
step 5A, database 26 of FIG. 1 is then examined to determine 
whether either of the corresponding terminals is defined as 
a recording terminal; that is, whether communication ses- 
sions initialed by the IP address of this terminal should be 
monitored. If true, then in step 6 A, the terminal status is set 
as a start session request from the terminal corresponding to 
the source port. 

Alternatively, the packet i s ex amined lo see if it is a 
connect packet in step 4B, which is determined according t o 
th e structure of the data in the packet . 'I'his structure is 
specified in the M. 225.0 recommendation, and includes at 
least the following types of information: 

protocolldcntifie r (the version of 1 1.225.0 which is 
supported); 

h245 Address (specific transport address on which M.245 
signaling is to be established by the calling cndpoint or 
gatekeeper); 

destination Info (contains an Endpoinf type to enable the 
caller to determine whether the call includes a gateway 
or not); and 

conferen celD (contains a unique identifying number to 
identity the particular conference). 

If the packet is a connect packet, t hen the second branch 
of the method is followed. In step 5B, the finu indicating th e -5 
t erminal stapis ^ is examined to determine if the tcrrnit|al 
status IS set as a start session request. In step 6B, the details 
of the call signal arc saved in a call progress database 78 of 
storage medium 30 (sec FIG. 2). Itiese d etajjs preferably 
include the source and destination IP addresses, the source 30 
and destination ports; the t ime at which the c ommunicatio n 
.s ession was initiale d, and any other relevant iniormation. In 
step 7B, the st atus of the terminal is set to "wait for the logic 
channel". 

^ It: the packe t has been <le lermin ed to be ;in H.245 p acket 35 
I l)y second filtering component 40 . the packet is passed to an 
11.245 c all control _m_oduIe 82 within m anagement modul e 
-28, as shown in FIG. 2. Such 11.245 packets are neccs.sary 
for 11.245 signaling. 11.245 signaling is cstabli.shed Ixslween 
two endpoints: an endt^oJni,, ai^fj Jl,/^;^^lt^;; poixlt^•oa^^ or 40 
an end point andjijGauy^ger (see FIGS. 6 and 7 below for 
examples and a description of such endpoints). Fach end- 

p oini is ca pable *^'j j^j^^!LnRJ^"fLpU^^'"?p ^**!lSf* '''"^ P'^^^ 
com mu n 'e\aljon sessiojr However, the system of the present 
invent iorT onl y mo t mors, ra_t 1 le r 1 1 la 1 1 i n i t ia t i t m , such com - 45 
municHtio ii ^-xsinns 1'hns, tlic system of the present inven- 
tion uses the 11.245 si gn a ling to determine when the com - 
nuinication ses.s jo njTas_siji rt ed i n orde r Ir.j: 1 1 ecu ve i y record 
t ne n^^^^f^^aia^ tJaf k^ I oLt hulstuauLc^muUalet L recon - 
s 1 rue ijg iuo t^Hielsessio n . .^o 

'llic .stejxs of the inaniigement method for 11.245 packets 
are ns follows, with rcrcrence lo FIG. 3U. In step I A of FIG. - 
3R, the 11.245 packet is examined lo delennine if it is^an 
o pen lo g ical channel request pack et. If it is, then in step 2 A, 
the te^ iiinajju i u us is examined to d etermine if the .siau is is 55 
"wait lor the logical channer'. If s<», then in step 3A the 
terminal status is .set to "wait l;or acknowledgment". 

Allernativcly, the 11.245 p a ckel i.se x a m i ned t o detcrm t n c 
if it is an open logical channet acknowteciginent p acket, as 
slunvii in step Hi. Ifuls, I lien in step 2B, t he tcrnnnal statues 60 
is exa nnnecj deterrni ne i£t.he si alus^is^wa i t fo r aelThowl- 
edgmenl". If so, then in ste|) 3B Ihe terminal status is set to 
"wail for lerniiiial capability", in step 4U, the transport 
address of the "called" or destination terminal is saved. This 
transport address is taken from the destination pori field 76 
of header 72 (see I'IG. 4H). It should be rioted that 11.225 and 
11.245 packets have identical header structures. 



Also alternatively, the 11.245 packet is examined to deter- 
mine if it is a terminal capability set packet, as shown in step 
IC. If Jt is, then in step 2C, the terminal capability is saved 
in call progress database « U (see i^iu. 2). In step 3(J,"ltie 
lefminai status^ is set to "in call procesS'T^UCh'"that the 
'communication^sc ssion has be en det enmined to be opened 
a nd such tha_t_ management mod uir28^an j?pw: receive 
data packets . ' 

If the packet has been determined to be a RTP packet by 
second, filtering com pone nt^O ^e packet is passed to a R AS 
(registration, admissions and statu.s) control module 84 
within management module 28, as shown in FIG. 2. 'line 
steps of I he management method for RTP packets are as 
follows, with reference to FIG. 3C. Ii Lsteu 1 of FIG. 3C, the 
t erminal status is examined to see if it is *'in call process". 
If so then in step 2, the RTP packets arc lia^iiiLya-a-BIP 
djit abas<;. 8.6 within storage medium 30 (sec FIG. 2). FIG: 4C 
shows the structure of the RTP packet header, which can be 
used to identify the communication session from which^the 
packet was taken. 

Finally, ifjhe.packet has biecn^ d eterrni n to be a RTCP 
pa cket by sec ond-filtering componenL 4U. the ^-iackot js 
passed ^o a RXCILcml roLmodu le ^ 8 8 within management 
module 28, as shown in FIG. 2. The steps of the management^ 
method for RTCP packets are as follows, with reference to 
FIG. 31). In step I of FICi. 3D, the t erminal status^ is 
* \QjI3ir!j-i! ^— ''^ ''IP ' all process". If so then in step 
2, the RTCP packets are saved in call progress database 80 
within storage medium 30 (see FIG. 2). FIG. 40 shows the 
stmcture of the RTCP packet header, which can be used to 
identify the communication session from which the packet 
was taken. 

Thus, FIGS. 3A-3D illustrate the method of the present 
invention with regard lo the tillering and storage of data 
l)ackets which constitute Ihe recorded communication 
session, as recorded by the system oC the present invention 
as shown in FIGS. I and 2. Of course, in addition to 
recording such communication ses.sions, the system of the 
present invention is alst> able to retrieve and to replay these 
communication sessioas to the user. Ilic_storedLco ni nju ni^ 
^V'^ij Q PrJ^.^?jQ" >~^^-^cg i?^s< ^Q f sto red data t:; a c kcts, _c a n , l^e 
retrieved a iKl dispU^^d^^ VIG. 2, in 

cj^njuh^onl^ilh^iUJ^^^ and video unit 36. The 

method of retrieving and replaying sessions of interest is 
shown ill I 'IG. 5, while cerlaiii other relevant portions of the 
system of the present invention are shown in FIG. 2. 

In st e|)_^i, of I'Ki. 5, the user inputs the information' 
concernmg the commiinicalion .session which is to he 
retrieved and replayed. Ill is information preferably includes 
the t erminal_nuinbe_i:^ or other designation information con- 
ce r ni ng a t le ast„onc-olLthe, part ies^xdiahe, com mujiiciulpn 
sesskm^lLinte^^^ the time at which the session started;' and 
the time at which the session ended. However, alternatively 
other information could be included in place of this 
information, as long assulllcienl information is provided for 
the euinmunication session of interest lo he identified. 

loilji^jj.of FIG. 5, call progress databa.se 80 (.see I'IG. 2) 
is searched by data restore unit 32 in order to lind the details 
of the connnunication session(s) in the specified lime range. 
These details are then com (ja red to the information entered 
by the u.ser to locate at least one communicalion .session of 
interest in the call range. 

In_stct L.3. R r_P_ _c!ataba.se_86 otj storage nic e! i n m (.see 
FKJ. 2) is searched, again by t l a i a re sto re ^ u n i t^^32 , to find 
substantially aimat^^li^Jviiti iVcmfTlie "at* least one commu- 
nication session in ttic specified call range 0|Hioiially and 
preferably, in step 4, if the audio potion communication 
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sgssion was recorde d in stereo, then the data packets are 
vOtMi^'^^A di vided into difl^/ent^aiidio channels . 

ln^step_5^ the data packets are y&toicc;! by data restore uq il 
3^by an ^£E,(Real Time Protocol) software modulijw^ 
within d ata restore unit_32 . RTF software module 91 orders 5 
the d ata packe ts within_eac, tij:J}aDiia l according to the time 

'^jlLPl-i^ • ^ shown in FIG. 4C, an IVI P packet 

hcaci er^fcal ures'sevcra 1 important lie Ids: a timcstamp lield 
94, a synchronization source f vSSRC) identifie rs field 96 and 
a cx)ntribmin^^SQurcc4JC:SRC)Jdcntiiicrs field 98. SSRC 
field 96 is used to det ermine the ^qurcc ^f^ thc,1^7y, P^^^^^ ts 
(the sender), which has a unique^idcnt it'y inR address TThe 
SSRC identifier). Hie CSRC identilicr in CSIiC fickl 98 is 
use d in a confercncc^ithjiniJ It ipte parties, and indicates the 
S § jUlidcn l j |i r,r f7 (; ,^ 11 ^ j' n'S Ti'mn^;! amp field 94 is used by 
in!li^Il,WArs„JIUlcy^ 91 to determine the relative tim e at 
which the da ta in each packet should be di spjayed. 

For example, preferably the audio stream data of the audio 
speech of one person is s ynchronized to that person's lip 
movement s as shown in the video str ea m , a process known 
as 'i ip synchrpni/ati on". Such synchronization requires 20 
more than simply replaying audio and video data at certain 
relative tirric points, since the audio and video data packets 
may not arrive at the same time, and may therefore have 
slightly dilVcrent timeslamps. 

Once the data packet |ias been correctly synchronizer! . the 05 
c( introl of th e d i splay of th e audio data is Uicn performed Iry 
a n'aucUo compcm lTr ^ data rcsto re _u'nitj3 2 according: to 

o ne or more audio CpJ>|S!Q's (see FIG. 2). The control of the 
display of the video data is then (x:r formed by a video 
c oniponent 10 4 of data restore imit 32 according to one or 
more video C ODEC'S (see FIG. 2). 

S uitable CpfjEC 's i nclude , but arc not limited to, an. 
a udio cocFec usinu CCl rt^ kecommendation G.71 1( 1988), . 
Pulse Code Modulation (1*CM) of voice frequencies; an 
audio codec using CCrPF Recxjmmcndation G. 722 (1988), 
7 kHz audio -coding within 64 kbit/s; a n^udio , c;Qc jec usi ng 
ril JJ_Rec_Qm mcndation QJ2XUUL^). Speech coders: 
Dual rate speech coder for multimedia communications 
transmitting at 5.3. and 63 Kbps; an audio codec using 
CCirr Recommendation 0.728 (1992), Coding of speech at 
16 Kbps using low-delay code excited linear prediction; an 40 
audio codec using ffU-T Recommendation G.729 (1996), 
Coding of speech at 8 Kbps using conjugate structure 
algebraic code -excited linear-prediction (CS-ACB1.,P); a 
video codec using I TU-T Recommendation IL26I (1993), 
Video codec for audiovi.sual services at px64 kbil/s; a video 4.s 
code using ri'U-'f Recommendation 11.263 (1996), Video 
codingfor low l>il rale communication; and substantially any 
other similar coding standard. 

As shown in M( 1. 2, the a udio dat a is displayed by audio— 
^ ^UiiLM' wliicli could include a I midspeaker . lV)r example. F>o 

</^^S^ ' '^^ video d:ita is displayed by video unit 36, which could 

inciude a display monitor screen, for example. vSiup 5 of 
I'lG. 5 is then preferably repeated, such that sul)siantially the 
entirety of the communication session is displayed. As 
shown in step 6, e ach data packet of Jhc__corumunicati on 55 
se ssion is^ exanviiieiLia-^e u.JJ Lthu ^ ca , l[ ^, b^ If the 

individual session has not completed, preferably step 5 is 
repeated. Alternatively and preferably, if iln^. t jyll lime is 
o ver, then call nroii rejsa_dat abtU3 « ^-J^ is searched to see if 
( Wher co nii nunicatitin sessions were recorded within the <m) 
p l»' jj^ ven Imie ^leriod, a s shown In step 7. 11 there is at least one 

file^r*' other such communication .session, then preferably the 

method o\' \ '\Ct. 5 is repeated, starting from step 2. 

According to preferred embodiments of the present 
invention, several configurations of the computer logging 
system .ire |)ossible, examples of which arc shown in I 'IGS. 
6 and 7. 



According to a first embodiment of the system of the 
present invention, shown in FIG. 6, a typical basic configu- 
ration system 1(M includes a single communication session 
management unit 13, substantially as shown in FIGS. 1 and 
2, according to the present invention. Communication ses- 
sion management unit 13 manages communication in a 
stand-alone intranet such as a I^N 106. I^N 106 is 
connected both to communication session management unit 
13 and to a plurality of tcrminalsJLQjS . designated as *' TI", 
"'17" and so forth, which Ibllow the H.323 protocol. IZach 
t e rm inaQ08Jsjaflj:ndnQ i^ t on LAN 106 which provides for 
real-time, two-way communications with another terminal 
108, a gateway 110, or a multipo i nt control unit 112. 'Ill is 

audio 



conimunicalion consists o t co u t ro 1 , 1 nd ic a 1 10 u s, 
streams, video streams, and/or data, l erminal 108 is-Qpli QO- 
ally o nljf_j^pablj^ p r oviding su c h commu^^ for 
aud t'oTgn ly, audio and data, audio and vTdeoToraudio, data 
a ncl~y]dcp . As noted previously in the "Description of the 
background Art" section, the M.323 entity could be a 
terminal which is capable of providing audio and/or video 
communication as a "LAN telephone", but could also be a 
stand-alone audio or video telephone. 

Gateway 110 (GVV) is constructed according to 11.323 and 
Ls an endpoint on I AN 106 which provides for real-time, 
two-way corrnnunications Ixjtwecn terniinaLs 108 on LAN 
106 and other suitable terminals on a WAN (not shown), or 
to another such Gateway (not shown). Other suitable termi- 
nals include those complying with Recommendations H.310 
(11.320 on IMSDN), 11.320 (ISDN), 11.321 (ATM), 11.322 
(GQOS-LAN), 11.324 (GS fN), I1.324M (Mobile), and V70 
(DSVD). 

Multipoint Control Unit (MCtJ^ lt2 is .in endpoint on 
LA N 106 which enables thEtic^ , or_niQre_Lcrm inals 108 and 
u atcways 110 to parJLLC. ipaiA:-iP»^LJDul,t,ij3Qipt ^^-Qpft^rcncc 

Preferably, system 104 also fealurcs a gatekeeper (GK) 
114, which is an 11.323 entity on LAN 106 which provides 
address translation and controls acce.ss to LAN 106 for 
terminals 108, gateways 110 and MCUs 112. Gatekeeper 
114 may alst> provide other services to terminals 108, 
gateways 110 and MCUs 112 such as bandwidth manage- 
ment and locating gateways 110. PrctAirably, gatekeeper 114 
enables the IP address of terminals 108 on LAN 106 to be 
determined, such that the correct IP address can be deter- 
mined *'on the fiy". 

In addition, LAN t06 may support non audio visual 
devices for regular'!' 1 20 data ;ipplical ions such as electronic 
whiteboards, still image transfer, lilc exchange, database 
access, etc. 

In basic system 104, a single, stand-alone communication 
se.ssion management luiit 13 is used for monitoring, Igggij ig 
iHid retrieval of all audio andA)r visual calls either between 



a ny two or more terminals 108 attached to l.,AN 106 or an y_ 
call to which nne of more of these turminals_l_0_8 is a narly . 
I lowever, lor the pre lei red emhodiment of the system of 
I'lG. 6 which includes gatekeeper 114, as well as for the 
system of V\G. 7, once the LT)m mimical ion session has been 
opened, preferably RAS control module 84 also performs 
RAS signaling between the management control module and 
NIC 16 where necessary for the ctuifiguration of the system. 
Such signaling uses 11.225.0 messages to perform regist 
at ion issions, bandwidth chan^es^ status, , and disengage 
I wc^jd urcsJie t w een nctoiolS-UiT^ rs . These mes- 

sages are passed on a RAS Signaling Channel, which is 
independent from Ihe Call Signaling Channel and the 11.24.S 
Control Channel. 1 1.245 open logical channel procedures are 
not used lo establish the RAS Signaling C'liannel. In LAN 
environments which contain a Gatekeeper (a Zone), the 
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RAS Signaling Channel is opened belween the endpoinl and 
the Gatekeeper. The RAS Signaling Channel is opened prior 
to the establishment of any other channels between H.323 
cndpoints. 

FIG. 7 shows a second embodiment of the system of the 5 
present invention as a zone configuration system 116. A zone 
118 Ls the collection of all terminals (Tx) 108, gateways 
(GW) 110. and Multipoint Control Units (MCU) 112 man- 
aged by a single gatekeeper (GK) 114. Zone 118 includes at 
least one terminal 108, but does not necessarily include one jq 
or more gateways i 10 or MCUs 112. Zone 118 has only one 
gatekeeper 114 as shown. However, in the prcl:erred emljodi- 
ment shown, zone 118 is preferably independent of LAN 
topology and preferably includes multiple LAN segments 
120 which are connected using routers (R) 122 as shown or 
other similar devices. 

Each monitored LAN segment 120 has a local commu- 
nication management unit 124 according to the present 
invention, of which two are shown. A central management 
unit 126 according to the present invention controls all local 
communication management units 124. In addition to cen- 
tralized database and control services, cen t ra 1 m a n age men t 
unit 126 can be u sed for the real-time monitorin)^ an d 
o IT- 1 i ne re s to ra t i 0 n,. 0 iLa u d i o n d/o r^vjdc Q„a3 oi mu , pica t i o n 
s essions f r onri^ j>Lnule_ iK>i n t . Central management unit 126 
is optionally and pret^crably either a dedicated unit similar in 
structure to local communication management units 124 but 
without the storage capability, or central management unit 
126 is alternatively and preferably integrated with local 
communication management units 124 to provide the tunc- 
tionality of both local communication management unit 124 
and central management unit 126 in a single station. Lt>cal 
communication management units 124 are preferably cither 
communication management units 13 substantially as 
described in FIGS. 1 and 2, or alternatively and preferably ,5 
are simpler units which lack the capability to retrieve and 
display a communication session locally. 

In still another preferred embodiment of the present 
invention (not shown), multi-u.scr operation based on Client / 
S erver architecture is preferably supported for basic system 
104 and zone systeniJJ6^ \n unlimited number of "Client" 
stations may be connected anywhere on the LAN, pjoyidin g 
users with management and m onitofjn fi/rctri eyalca ^ 
^^^l ^X^Jj ijlj^^UzyJ ilVUUillKlO o f^ a'C h spe c i 1 ic user. 

It wi 1 1 be a p [ircci at ccl t li a I the aljove ' "lescripfiSiii^rafc 
intended only to serve as examples, and that many other 
enihtidiinents arc possible within the S{)irit and the .scope of 
the present invention. 

What is claimed is; 

1. A system for managing a computer network -based 
telephone session over a computer network, the computer 
network being divided into a plurality of segmenis, the 
system cotnprising: 

(a) a nciwork comiector for connecting to the computer 
network and for r eceivinti data packe ts IVom aj^ingle 55 
segment of the computer network; 
('^) ^'jjh^xmil^UiVt for lihering said data i:)acket s from said 
si ngle setj , [|icj it and for a ccepting said data packe ts 
s ubstanliallv only if said data packets coiUain data 
selected from tlie ii rmq) ^ consi.stij i^ of a udio data and h*) 
vi deo d ata, such that said data packets form at least a 
|)art 0^1 he computer netvvork-l?ased telephone session 
and such that said dala packets are selected data pack- 
els; 

(c) a mana i^cmeni unit for rm^mng said selected data 6.*; 
packets Irom said single segment and for storin g said 
selected data packet s, such that said s elected cki ta 
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pack ets are stored daia packets, wherein said manage- 
ment unit and said fiUerina uni t form a local man age - 
mem unit for said single segment of the computer 
network, said local management unit analyzing said 
s elected dala packets if the computer net work -based 
telephone session occurs within said single segment; 

(d) a storage medium for receiving and storing said stored 
data packets IVom said local management unit, such 
that at least a portion of the computer net work -based 
telephone session is stored; and 

(e) a central management unit for controlling each local 
management unit, said central management unit con- 
trolling storage in said storage medium and said central 
management unit analyzing said selected dala packets 
from the computer network-based telephone session if 
the computer network-based telephone session includes 
data packets transmitted on a plurality of segments of 
the computer network. 

2. The system of claim 1, further comprising: 

(f) a data restore unit for retrievin >^ and displaying said at 
l east a portion of the computer network-based tele- 
phone session, .said data^ re store unit requesting said 
d ata packets from said storae,e me dium^thcou tih said 
c entral management u n iu and said d ata restore, u nit 
re CO n st ciicllmL sa id d a t a pac kc t s f or displayipL> sai d at 
least a portion of the computer network -based tele- 
phone session. 

3. The system of claim 2, wherein said dala restore unit 
further compri.ses a communication .session di.splay unit for 
displaying at least a portion of the computer network -based 
telephone se.ssion. 

4. The system of claim 3, wherein said communication 
session display unit is .selected from the group consisting of 
a video unit and an audio unit. 

5. The .system of claim 2, further comprising: 

(g) a databa.se connected to said liltering unit for storing 
filtering intormation, said fdtering information includ- 
ing at least one IP address of a party whose computer 
network-based telephone .sessions are monitored; 

wherein said filtering unit accepts said data packets accord- 
ing to .said fdtering information, such that said filtering unit 
sufistantially only accepts said dala packets if said data 
packets fulfdl said liltering information. 

6. The system of claim 5, further comprising: 

(g) a user computer for receiving at Ica.si one command of 
a user ;uid for displaying information to .said u.scr, such 
that .s;ii(l user determines .said lilturirig iiiforniatioM 
according to said at least one command of .siiid u.ser. 

7. The syslem of claim 6, wherein the computer network 
is selected from the group consisting of a LAN (local area 
network) and a WAN (wide area network). 

8. The .system of claim 7, wherein the computer ticlw<)rk 
is a LAN (local area network). 

9. The system of claim 1, wherein .said network connector 
is a network interface card. 

10. A method for storing at least a portion of a computer 
uetvvork-l>a.si:d telephoni: .scssiori performed on a computer 
network, the computer network -based leleplione session 
being performed lx;tween a packet source and a packet 
destination. I he steps of the method lx:ing performed l)y a 
dala proces.sor, the melhod comprising the steps of: 

receiving a dala packet from the paclxcl source on the 
computer network; 
(b) analyzing said data packet to determine if .said dala 
packei is a computer nelwork-l)a.sed telepfione se.ssion 
packet; 
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(c) if said data packet is said computer network-based 
telephone session packet, filtering at least data in said 
data packet to determine if said data includes computer 
network-based telephone session data; 

(d) if said data includes computer network-based tele- 
phone session data, analyzing said computer network- 
based telephone session data; and 

(e) storing said computer network-based telephone session 
packet to form a stored packet according to said type, 
such that said stored data packet forms at least a portion 
of (he computer network-based telephone session. 

11. The method of claim 10, wherein said data packet has 
a header and the step of analyzing said data packet in step 
(d) further comprises the step of: 

(i) filtering said header of said data packet to retrieve 
header data related to the computer network-based 
telephone session. 

12. The method of claim 11, wherein substep (i) of step 
(d) further comprises the step of: 

(1) analyzing said header data to determine if said data 
packet Ls an IP packet. 

13. 'nie method of claim 12, wherein the step of analyzing 
said header data in substep ( 1) further comprises the steps of: 

(i) examining said header of said IP packet to determine 
an IP address of sait! packet source; 

(ii) determining if said IP address is a recorded IP address; 

(iii) passing said IP packet to form a passed IP packet 
substantially only if said IP address is said recorded IP 
address; and 

(iv) alternative iy. dumping said IP packet. 

14. I'hc method of claim 13, wherein the step of deter- 
mining if said IP address is said recorded IP address is 
performed by comparing said IP address to a list of IP 
addresses from packet sources, such that if said IP address 
is included in said list, said IP address is said recorded IP 
address. 

15. The method of claim 13, wherein step (d) further 
comprises the steps of: 

(ii) analyzing said IP packet to determine whether said 
passed IP packet Ls an H.225 packet, a H.245 packet, an 
R TP packet or an Rl'CP packet; 

(iii) if said type of said passed IP packet is said 11.225 
paekcii, dctcrininiiig whet tier said 11.225 packet is a 
setup pJickci or ;i connect p;ickel; 

(iv) if said 11.225 packet is said setup packet, .setting a 
stains Hag as "start session requt^sl"; 

(v) alternatively, if said i 1.225 packet is said connect 
packet and said status flag is "start session request", 
.^iioring at least one detail of the computer network- 
i<ased lelepliatie session; and 

(vi) sotting said status flag as "wait tor logic cliauiicl". 

16. The method of claim 15, wherein step (d) liirther 
comprises ttie steps of: 

(vii) alternatively, if siiid type of said passed IP packet is 
.said 1 1.245 packet, determining whether 11.245 packet 
is an open logical ctiaiuiel request packet, an open 



logical channel acknowledgment packet or a terminal 
capability set packet; 
(viii) if said H.245 packet is said open logical channel 
request packet and said status flag is **wait for logic 
channel", setting said status flag as "wait for acknowl- 
edgment"; 

(xi) alternatively, if said H.245 packet is said open logical 
chanel acknowledgment packet and said status flag Ls 
"wait for acknowledgment", performing the steps of: 

(A) setting said status flag as "wait for terminal capa- 
bility"; and 

(B) saving a transport address of the destination of the 
communication session; and 

(xii) also alternatively, if said H.245 packet is said ter- 
minal capahility .set packet, performing the steps of: 
(A) storing a capability of the packet destination from 

said terminal capability packet; and 
(13) setting said status Hag as "in call progress". 

17. The method of claim 16, wherein if said status flag is 
"in call process" and said type of said passed IP packet is 
said RTP packet, storing said RTP packet. 

18. Ttie method of claim 16, wherein if .said status flag is 
"in call proces.s" and said type of said passed IP packet is 
said RTCP packet, storing said RTCP packet. 

19. The method of claim 10, further comprising the .steps 

of: 

(f) retrieving said stored data packet to form a retrieved 
data packet; and 

(g) reconstructing at least a portion of the computer 
network-based telephone .session according to said 
retrieved data packet. 

20. The method of claim 19, wherein the step of retrieving 
said data packet of step (f) includes the steps of: 

(i) retrieving a source IP address of the packet source, a 
.start time of tfie network-ba.sed telephone session, and 
an end time of the computer network -based telephone 
session; and 

(ii) .selecting at least one computer network-based tele- 
ptione session according to said source IP address, said 
start tinic and said end time. 

21. 'i'he method of claim 19, wherein the .step of recon- 
structing at least a portion of the computer network-based 
telephone session of step (g) includes displaying audio data. 

22. The method of claim 19, wherein the step of recon- 
slaicting at least a portion of the computer network-basetl 
telephone session of step (g) includes displaying video data. 

23. The method of claim 19, wherein the .step of recon- 
structing at least a portion of the computer network-ba.sed 
telephone .se.ssion of step (g) finltier comprises the steps of: 

(i) receiving substantially only RTP packcis; 

(ii) cxanMiiiing a header of said R I P packets to determine 
a lime stamp lor each of .said R I P packets; and 

(iii) displaying .said RTP packets in order according to 
said lime stamp. 
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A method and apparatus are used in a gateway to discard 
selected frames received with a selected encodcd- 
information-type from a communication link with a larger 
bandwidth to avoid overflowing an internal delay variance 
removing queue used for protocol traaslaiion to a commu- 
nication link with a smaller bandwidth. Vhc discarded 
frames do not decrease the quality of translated information. 
A visual delay variance removing queue congestion indica- 
tor is included to indicate three levels of congestion in the 
delay variance removing queue for received frames. The 
method and apparatus are used in a multimedia gateway 
which LS translating audio/video conferencing protocols 
(e.g., 11.320, H.323/LAN 11323/FPP and H.324) received 
from a communication link with a large bandwidth and sent 
to a communication link with a smaller bandwidth. 
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FIG. 14C 
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MKTHOO AND APPARATUS FOR ADAR IVK 
PRIORITIZATION OF iVlULTIPLE 
INFORMATION TYPKS IN HIGHLY 
CONCKS I FI) COMMUNICATION DEVICKS 

FIELD OF INVENTION 

ITie present invention relates to communication in com- 
puter networks. More specifically, it relates to adaptive 
prioritization of multiple information types in highly con- 
gested corn nuinicat ion devices. 

BACKGROUND OF HIE INVENHON 

As is known in the art, a variety of computing devices arc 
often connected together to form a computer network. 1^e 
computer network may be a Local Area Network (**LAN") 
that connects devices over a small geographical area, or a 
Wide Area Network (*'WAN") that connects devices over a 
large geographical area. The computing devices include 
video cameras, CD-ROMs, microphones, televisions, 
computers, modems, cable modems and other devices that 
send high resolution images, graphical images, moving 
images, audio and data in addition to textual information. 
Dilfcrent types of computer networks may be interconnected 
to each other to form larger computer networks (e.g., the 
Internet), llie interconnections include LAN-LAN, I.AN- 
WAN, WAN-WAN, LAN-WAN-LAN, and other network 
interconnections. 

The computing devices transfer multimedia information 
(e.g., audio, video and data) between two or more computer 
networks. Transferring multimedia information between two 
computer networks may or may not require a reserved bit 
rate transmission capacity, and a reserved bandwidth possi- 
bly for the total duration of the transaction. For example, 
multimedia information for a ten second video clip with 
.sound that is being sent between two points in a Ethernet 
LAN, requires a sign ilka fit portion of a 10 Mega -bits- per- 
seco nd ("'Mbps'*) data transmission capacity available on the 
LAN for ten seconds to send the multimedia information. 

Gateways connect computer networks using ditfercnt 
network protocols operating at ditferent transmission 
capacities. For example, a gateway may have network 
connections to serial data lines connected to one or more 
modems. The serial data lines may be used one at a time at 
a relatively low transmission speed (e.g., 14,400 bps, 28,80(1 
bps, or 56,000 bps) or can he bundled into a group at a higher 
transmission speed. In contrast, the gateway may alst* have 
one or more LAN connections (e.g., Ethernet, Token Ring, 

Fiber Distrihuled Data Intcrrace ("FDDI")). The LAN 
connections are higher .speed connections (e.g., 10 Mbps) 
and are shared among fnulti]}le devices. 

'i1ic gateway tr;inslatcs information contained in a first 
protocol being u.sed on a first network connec''on rnto a 
second prottKJol l)cing used on second network connection, 
and visa-versa, without undue delay or loss of information. 
I'br example, a modem t)perating at 28,800 bps may be ii.sing 
the International Telecommunications Union- 
'lelecommiinication vStandardi/ation Sector (*'rrU-'T\ for- 
merly known as the CCI IT) 11.324 audio/video conferenc- 
ing protocol and a I ,AN operating at 10 Mbps may he ii.sing 
the I rU- T i!.323 audio/video ct)nferencing protocol for a 
video conferencing connection. A gateway translates 11.323 
from the I AN into 11.324 tor use on the modem, and 
visa -versa, witfioiii iiimImc delay or loss of in form:) I ion even 
though the LAN is iransmilting information at 10 Mbps and 
the modem is transmitting information at 28,8(Xl bps. 

However, the gateway may also translate H.323 oti a LAN 
to 11.323 on a serial line using the Point-to-Poini Protocol 
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f PPP"), H.323 on a LAN to H.320 on an Integrated 
Services Digital Network ("ISDN") line, or traaslatc H.32x 
on a LAN or a serial line to H.32x on a LAN or a serial line, 
llic gateway is also responsible for maintaining timing 

5 relationships and packet sequencing even though a first 
protocol may use timing relationships and a second protocol 
may not use timing relationships. 

When gateways are used to translate multimedia infor- 
mation between two computer networks, logical multimedia 

1*^ channels are typically created with separate audio, vicleo and 
data channels, llie audio and video channels are typically 
allocated with predetermined, fixed maximum bandwidth. 
For example, on a modem connection a audio channel may 
have a bandwidth of 5,300 bps and a video channel may 

15 have a bandwidth of 23,500 bps for a multimedia bandwidth 
of 28,800 bps. A LAN connection may use audio and video 
channels with larger bandwidth allocations since the I AN is 
capable of transmitting information at a much larger overall 
multimedia bandwidth (e.g., 10 Mbps). 

"'^ There are several problems associated with using gate- 
ways or other internetworking devices known in the art to 
interconnect computer networks operating at different trans- 
mission capacities. For example, the logical channels for 
larger bandwidth computer network connect ioas (e.g., LAN 
connection.s) are often connected to logical channels for 
smaller bandwidth computer network connections (e.g., 
modem connections). The larger banchvidth connections 
have no way of determining they are connected to smaller 
bandwidth, and more constrained connections. This will 
cau.sc a constant congestion problem on a gateway since the 
logical channels for the larger bandwidth network is con- 
stantly transmitting more inlbrmation than can be accepted 
by the lower bandwidth network. In addition, the gateway 
must translate between two or more dilTerent protocols for 
the connections without undue delay or loss of information. 

If data is sent along with the audio and video information 
on a multimedia connection, the congestion problems are 
further aggravatetl on the gateway. The gateway allocates a 

^.^ chunk of transmission bandwidth for a logical data channel. 
Since the data transmission is typically very bursty, the 
transmission bandwidth allocated for data channel is often 
wasted when no data is being sent. For example, the gateway 
may allocate a 5,000 bps logical data channel for each 
nelwork connection using the data, {""or a modem with a 
bantlwidth of 28,8(KI bps, this wastes about 17% of the 
available bandwidth on the modem. This is a significant 
wa.'-.tc of bandwidth on a smaller bandwidth network con- 
nection. 

51) Another problem is lhat the gateway inusl maintain timing 
relationships and packet sequencing translating between 
certain protocols. The timing relationships and packet 
sequencing must he maintained even though a first prtitocol 
uses timing and a .second protocol does not. 

'^'^ SUMMARY OF THE INVENTION 

In accordance with a preferred embcxliment of the present 
invention, the congestion problems for translating protocols 
are overcome. A method and apparatus lor adaptive I y pri- 

00 oriti/.ing between two or more encoded-informat ion-types 
received over a cxmimunication link in multiple frames in an 
internetworking device (e.g., a gateway) is provided. The 
frames include multiple data bits. The method includes 
receiving multiple frames for a first network protocxd over a 

(i.s first communication link having a first communication band- 
width, rite nuihiple frames have one of ninltiple ol'encoded- 
in formation-types. 
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ITie received IVames are stored in a delay variance remov- 
ing queue in memory in the internetworking device. The 
delay variance removing queue allows the internetworking 
device to compensate for sequencing or liming relationships 
used in the first network protocol. The delay variance 
removing queue is also used as a buffer if it is not necessary 
to maintain timing relationships between two similar pro- 
tocols. The frames from the delay variance removing queue 
arc translated into a second network protocol. Periodically a 
test is completed to determine if a number of frames arriving 
on the first network connection exceeds a predetermined 
queue congestion threshold. The delay variance queue can 
be overflowed if the first communication link has a larger 
bandwidth than the second communication link. If the queue 
congestion threshold is exceeded, an cncoded-information- 
type encoded in a frame is selected to store in the delay 
variance removing queue. Received frames are discarded 
that do have the selected encoded-in formation-type until 
enough frames have been processed so the delay variance 
removing queue has reached a predetermined length. The 
translated frames for the second network protocol are sent 
over a second communication link having a second com- 
munication bandwidth, the second communication band- 
width being less than the first communication bandwidth. 

In another embodiment of the present invention, received 
frames that do not have the selected encoded-informalion- 
lype are discarded and not stored in the delay variance 
removing queue. In yet another embodiment of the present 
invention, a new cncoded-inforrnat ion-type may be selected 
for a specified lime period (e.g., to send data) ;ind all f rames 
that do not have the new cncodcd-informat ion-type arc 
discarded during the specified time period. 

The method and apparatus allow received frames with a 
selected-informalion-type to be discarded in a multimedia 
gateway without significantly affecting the quality of infor- 
mation when a first network protocol (e.g., M.323/LAN) is 
translated into a second network protocol (e.g., H.320, 
H.323/PPP, H.324, II.32x, etc.). 'Ihc discarding of informa- 
tion is often necessary since frames are arriving on a 
communication link with larger bandwidth fa.ster than they 
can be translated and sent on a communication link with 
smaller bandwidth. 

The foregoing and other features and advantages of a 
preferred embodiment of the present invention will be more 
readily apparent from the following detailed description, 
which proceeds with references to the accompanying driiw- 
ings. 

BKIIil' IJBSCRIPriON 01' TMB DRAWINGS 

I'lO. I is a block diagram of a computer network used to 
impiemeni a preferred embodiment of the present invention; 

rKivS. 2A and 2H are a How diagram illu.straling a method 
for adaplivcly priori) i/Jiig between two or more information 
types received on a communication link; 

ri(!i. 3 is block <!iagram illustrating a packet synchroni- 
zation in an internetworking device; 

1*1 G. 4 is a l>lock diagram illu.strating a delay variance 
removing queue in an internetworking device; 

I'lG. 5 is a tlow diagram illustrating a lirsl method of 
discarding frames from a delay variance removing queue; 

I'lG. 6 is a block diagram illusl rating the method of I'lG. 

5; 

I'lG. 7 is a Mow diagram illustrating a second method of 
discarding frames from a delay variance removing queue; 
\ \G. H is a block diagram illii.slraliiig the rnetftod of T'lG. 

7; 



FIG. 9 is a flow diagram illustrating a method for dis- 
carding received frames; 

FIG. 10 is a block diagram illustrating the method of FIG. 

9; 

^ FIG. 11 is a flow diagram illustrating method for receiving 
frames with a new selected information type; 

FIG. 12A is a block diagram illustrating a protocol 
translation system; 

to FIG. 12 B is a block diagram illustrating another protocol 
translation system; 

FIG. 13 is a block diagram illustrating a visual queue 
congestion indicator; and 

FIGS. 14A, 14B and 14C are a flow diagram illustrating 
15 a method for displaying a visual indication of congestion in 
a queue. 

DETAILED DESCUIFI ION OF A PREFERRED 
EMBODIfVIENT 
20 Protocol Translation System 

FIG. 1 is a block diagram of a computer network 10 u.sed 
to implement a preferred embodiment of the present inven- 
tion. Computer network 10 includes a first computer net- 
work 12 and a second computer network 14 interconnected 
25 l>y an InterNet working Device 16 ("INl)''). However, more 
or fewer computer networks could be interconnected by IND 
16 and the invention Ls not limited to interconnecting two 
computer networks. In addition, computer network 10 may 
include additional network devices (i.e., other than INI) 16) 
.^0 and additional network nodes which are not shown in FIG. 
I. 

IND 16 Ls also called an "IntcrWorking Unit" ("IWU"), an 
'intermediate System" (*iS") or a "gateway." IND 16 has 
multiple communication links 18 to first computer network 

35 12 and nuiltiple conununication links 20 to second comfiuter 
network 14 (illustrated as single connections 18 and 20 in 
FIG. 1). There may aLso be multiple virtual communication 
channels over the communication links (18, 20) (e.g., sepa- 
rate virtual channels for audio, video and data information). 

40 IND 16 has a software server 22 with a listing of network 
protocols 24 for first computer network 12 and a listing of 
network protocols 26 for .second computer network 14. IND 
16 uses software server 22 to translate network protoa)ls 
that arrive on a communication link from one computer 

45 network imo a protocof for anotfier computer network 
without undue del;iy or loss of infnrmntion. 

I'or example, a first network protocol PI that arrives over 
a first communication link 18 with a fi i:st communication 
bandwidth (e.g., 10 Mbps) from first computer network 12 

50 is translated with software server 22 into a second network 
protocol 1^ for .second computer network 1.4 using protocol 
listings 24 and 26. Protocol P2 is sent over a second 
communication link 20 to second computer network 14. 
Second communication link 20 may have a second commu- 

55 nieation bandwidth (e.g., 28,800 bps) that is smaller than the 
first comnuinication b;indvvtdth but protocol P2 is sent 
without undue delay or loss of information. 

An operating environment for IND 16 of the present 
invention includes a processing system with at least one high 

60 Speed Central Processing Unit C'CJPU"), in conjunction with 
a memory system. Although described with one CPU, alter- 
natively multiple CPUs may be u.sed. 

The memory .system includes main memory and second- 
ary storage. The main memory is high -.speed Random 

05 Access Memory (''RAM") and Read Only Memory 
C'kOM"). Main memory can include luiy additional or 
alternative high-speed memory device or memory circuitry. 
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Secondary storage takes the form of long term storage, such 
as ROM, optica! or magnetic disks, organic memory or any 
other volatile or non-volatile mass storage system. Those 
skilled in the art will recognize that the memory system can 
comprise a variety and/or combination of alternative com- 5 
poncnts. 

In accordance with the practices of persons skilled in the 
art of computer programming, the present invention is 
described below with reference to acts and symbolic repre- 
sentations of operations that are performed by the processing lo 
system, unless indicated otherwise. Such acts and operations 
are referred to as being '^computer-executed" or "CPU- 
executed." 

It will be appreciated that the acts and symbolically 
represented operations include the manipulation of electrical 15 
signals by the CPU. The electrical system represent data bits 
which cause a resulting transformation or reduction of the 
electrical signal representation, and the maintenance of data 
bits at memory locations in (he memory system to thereby 
reconfigure or otherwise alter the CPU's operation, as well 20 
as other processing of signals. The memory locations where 
data bits are maintained are physical locations that have 
particular electrical, magnetic, optical, or organic properties 
corresponding to the data bits. 

Hie data bits may also be maintained on a computer 25 
readable medium including magnetic disks, optical disks, 
and any other volatile or no n- volatile mass storage system 
readable by the computer. The computer readable medium 
includes cooperating or interconnected computer readable 
media, which exist exclusively on the processing system or 30 
be distributed among multiple interconnected processing 
systems that may he local or remote to the processing 
system. 

Adaptive Prioritization of Multiple Information Types in a 
Protocol 35 

FIGS. 2 A and 2B are a How diagram illustrating a method 
28 for prioritizing between two or more cncodcd- 
information-typcs tor a selected network protocol received 
in frames over a communication link on an internetworking 
device such as IND 16. The Irames include multiple data 40 
bits. At step 30 in FIG. 2 A, multiple frames arc received for 
a Hr.st network protocol over a first communication link 
having a first communication bandwidth. The multiple 
frames include multiple encoded-in form at ion-types (e.g., 
vitloo cotlec frames, audio cckIcc frames and data tVames). 45 
l"or selecled protocol translations (e.g., M.323-'-'ll.324) I hat 
require timing be maintained, received frames are stared in 
a Delay Variance Removing ('M)VR") queue in memory in 
the internetworking device at step 32. The DVR queue 
allows IND 16 to compensate for packet sequencing and 50 
timing relationships during protocol translation. The frames 
I'rom the DVR queue are translated inio a sticond network 
protocol at step 34. In a prelerred embodiment of the present 
invention, the second communication bandwidth is less than 
the first communication bandwidth but the translated frames 55 
arc sen! withoni nndue delay or loss of information. 
However, the C(»mmunication band widths may also be 
equivalent un both the first and second communication links. 

Periodically, a test is completed at step 38 (FIG, 2B) to 
determine whether a number of frames arriving on the first 60 
network connection exceeds a predetermined queue conges- 
tion threshold. IT the congestion threshold is exceeded, an 
encoded-inforniat ion-type encoded in a ('rame is selected at 
step 40 to store in the DVR queue. Frames that do not have 
the selected encoded-inl'ormation-type are discarded at step 65 
42 nnlil eiuiugli Irames in the OVR queue are processed so 
the DVR queue reaches a predetermined length. When 



enough frames in the DVR queue are processed so the DVR 
queue reaches a predetermined length at step 38, discarding 
of frames that do not have the .selected encoded-in format ion- 
type is discontinued at .step 44. Translated frames for the 
second network protocol are sent over a second communi- 
cation link having a second communication bandwidth at 
step 36 (FIG. 2A). 

The first and second communication links (18,20) in 
method 28 may also include two or more virtual channels. 
For example, the two or more virtual channels may include 
a first virtual channel for audio information and a second 
virtual channel for video information, and a third virtual 
channel for data information. 

In another embodiment of the present invention, if no 
timing compensation is necessary, then the received frames 
are stored in a DVR queue u.sed as a butfer. When the DVR 
queue is used as a buffer, no timing compensation is com- 
pleted in the DVR queue. The buffer is used to allow a 
protocol from a larger bandwidth connection to be tran.slatcd 
into a protocol used over a smaller bandwidth connection 
(e.g., H.323/LAN^H.323/PPP). 

For an embodiment u.sing the DVR queue as a buffer, 
returning to FIG. 2 A at step 30, multiple frames arc received 
for a first network protocol over a first communication link 
having a first communication bandwidth. Ilie frames are 
stored in the buffer at .step 32. 'IVie frames stored in the buffer 
are translated into a second network prottxol at step 34. 
Periodically, a test is conducted at step 38 (FIG. 2B) to 
determine whether a number of frames arriving on the first 
network connection exceeds a predetermined buffer cxinges- 
tion threshold. If the buffer congestion threshold is 
exceeded, an encoded- in format ion-type encoded in a frame 
is .selected at step 40 to store in the buffer. Frames that do not 
have the .selected encodcd-in formal ion-type are discarded at 
step 42 until enough frames in the buffer are processed so the 
buffer reaches a predetermined length. When enough frames 
in the buffer are processed so the buffer reaches a predeter- 
mined length at step 38, discarding of frames that do not 
have the selected encoded -in Ibrmat ion-type is discontinued 
at step 44. Translated frames for the second network proto- 
col arc sent over a second communication link having a 
second communication bandwidth at step 36 (FIG. 2A). 

As is known in the art, the Open Systems Interconnection 
("OSl") model is used to describe computer networks, fhe 
OSl model consists of seven layers including lYom lowcst- 
to- highest, :i phy.sical, ihila link, network, transport, session, 
application and presentation layer. The physical layer trans- 
mits bits over a communication link. The data link layer 
transmits error free frames of data. The network layer 
provides internetworking, determines routing information 
and controls a communications subnet. A cx)mmunications 
subnet is a collection of transmission media required Ibr 
routing and data transmission. Most INDs (e.g., gateways) 
operate at all levels in the OSl model. 

Standards organizations such as the International Tele- 
communications Union-Telecommuniealion Standardiza- 
tion Sector ('M rU- T", formerly known as the CCTIT), the 
Institute of Flectrical and Electronic Lngineers ("IBBB"), 
International Organization for Standards ('iSO") and others 
establish recommendations for standard network protocols. 
Devic-es connected to computer networks such as INDs 
typically use standard protocols as well as proprietary pro- 
tocols lo communicate with other devices in a computer 
network. INDs identify Ixnh .standard and proprietary net- 
work protocols. 

In one emlxuliment of the present invention, method 28 is 
used lo translate audio/video conferencing protocols in a 
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multimedia gateway (e.g., IND 16) that are received for 
audio/video conferencing. First communication link 18 is a 
LAN communication link with an exemplary handwidtli of 
10 Mbps. Second communication link 20 is a serial com- 
munication link via a modem with an exemplary bandwidth 
of up to 56,0(10 bps. However, the invention is not limited to 
the exemplary bandwidths, and other bandwidths may also 
be used for the first and second communication links (18, 
20). 

Method 28 is used in IND 16 with a DVR queue or with 
a DVR queue as a bulTcr for translating between ITU-T 
1 1.323 and H.323 (e.g., H.323/LAN and H.323/1^PP), H.323 
and ITU-T 11.324, 11.323 and ITU-T 11.320 or ITU-T H,32x 
protocols, where "x*' represents a protocol in the ITU-T 
11.320 scries. As is known in the art, ITU-T 11.324, entitled 
"■ Terminal for Low Bit Rate Multimedia Communication" is 
the ITU-T recommendation for standard videoconferencing 
using Plain Old Telephone Service ("POTS") lines. 11.324 
uses \ W- V 11.223 for its data link layer. 11.223 is entitled 
"Multiplexing Protocol for l.x>w Bit rate Communications." 
riV- r 11.323, entitled "Visual Telephone Systems and Ter- 
minal Equipment for Local Area Networks That Provide a 
Non-Guaranteed Quality of Service, " and 11.323 version-2, 
entitled "Packet Based Multi-media Communication 
Systems," arc ttie main i-aniily of video conferencing rec- 
ommendations for Internet Protocol ("IP") networks. 11.323 
uses H.225, entitletl "Media Stream Packetization and Syn- 
chronization on Non-Guaranteed Quality of Service LANs" 
or H.225 vcrsion-2 entitled "Call vSignaling Protocols and 
Media Stream Packet izat ion for Packet Based Multi-media 
Communication Systems" for its data link layer. ITU-'T 
1 1.320, entitled "Narrowband Visual Telephone Systems and 
Terminal Bquipment," is a standard protocol for videocon- 
ferencing using ISDN or similar telephone circuits. 

For example, IND 16 using method 28 will identify I L323 
sent by first computer network 12 over communication link 
18 (a LAN communication link), translate H.323 data into 
11.320, 11.324 or M.32x and send it to second computer 
network 14 over communication link 20 (a serial commu- 
nication link, or a LAN communication link). IND 16 could 
also identify 1 1.323 from second computer network 14 sent 
over communication link 20, translate H.323 data into 
11.320, H.324 or ll.32x data, and .send it to first computer 
network 12 over communication link 18. Inrst computer 
network 12 and .second computer network 14 would then 
h;ive a I wo -way audio/video conferencing com mnnica lion 
patli. 

During an audi*>/video conferencing call, audio informa- 
tion is typically supplied by audio Input/Output ("I/O") 
equipment (e.g., a microphone/speaker) thai uses an audio 
coder/decoder ("codec") to capture audio iti format ion. 
Audio codecs ;ire kiiuwn lo those skilled in the art and 
include G.7I I, ('..722, 0.723.LLI, G.72<S and C;.729. 
However, other audio codecs could also be u.sed. 

Video information is captured by video I/O equipment 
(e.g., a video cairiera) that uses ;i video codec. Video codecs 
are known to tho.se skilled in the art and include 11.26 f ;md 
I I. 263 video codecs. However, other video cctdecs could also 
be used. TTU-'T 1 1.261, entitled "Video Codec for Audiovi- 
sual Services at Px64 Kbit/sec," de lines video coding ba.sed 
t)n 1^- number of 6^,(KK» bps channels (e.g., ISDN 6'IK bps 
li-channeLs) where I* is two or more, 11.26 1 is a video coding 
method designed for 11320, 11.323 and other H.32x proto- 
cols. ITU-T 11.263, entitled "Video Coding for Low Bit Rate 
Communication" is a video coding method u.sed for 11.323, 
11.324. and other 11. 32x protocols. 11,263 uses ihe techniques 
of 11.261 plus significant enhancements. For example, 11.263 



uses half pixel prccLsion for motion compensation while 
11.261 uses full pixel precision and a loop filter. H.263 
supports five resolutions including Quarter Common Inter- 
change Formal f'QCIF"), Common Interchange Format 
C'CIF"), 4CIF, 16CIF, which are four and sixteen times the 
resolutions of CIF, and Sequenced Quarter Common Inter- 
change Format ("SQCIF"). H.261 supports only QCIF and 
CIF. 

1 1.263 uses three types of video codec frames as is shown 
in Table t below. 

TABLF 1 



40 



11.263 Codec 
l''rnmc 'types 



l>;scri|jtion 



25 



l-luanie I-fraiitcs nic iiitrn fiamcs which arc encoded 

similarly to Joint Picture lixpcrt Group 
("JI'liG"). 'llii.^ nienns, no motion 
compeiisaliod prediction is used. 'ITiesc 
frames are sent |5eriodically (e.g., every 
second) to enable n decoder to recover from 
errors. The frame size of an [-frame can be 
5K bits to 50K bits for Quarter tronimon 
[nterchange l^omiat ("QCIt'") and larger for 
Common Interchange I'ormat ("CII'""). 

P-l'ramc P- frames arc error values for the motion 

compensation prediction process that is used. 
If the motion prediction process is good, P- 
frames consist of motion vectors with no error 
data. If the motion compensation prediction 
prx)cess Ls bad the error data is usually 
between ItKl hits and lOK bits of data. 

B-[*rame IJ-fmmcs arc predicted using a previous I- 

fmnic or P-framc and a current P- frame, 'llie 
main use of li- frames is the ability lo increase 
the frame iTitc with only a minimal amount of 
incrca.sc in the bit rate. B-framcs arc not sent 
alone but are combined with P- frames. 

Pli-Frame PB-frames arc I'-franies jointly encoded with a 

ii-t'mnic. 'I"hc Pli-framcs are twx) fnimcs and 
there is a delay of one frame involved. PB- 
fmnies average about 200 bitR to 20 K bits of 
datn. 



P, and FB frames are used to lower the total amount of bits 
transmitted to display a video image by sending motion 
compensation prediction errors, l-framcs arc sent |)eriodi- 
cally to recover from the motion compen.sat ion prediction of 
the P, and PB frames. Sending l-t"rames is necessary but not 
desirable since the l-fiaine ctuitains bits representing tlie 
whole video image while P and PB frames contain bits 
representing only changes in the video image since the last 
frame (1, P, or PB) was sent. Vor example, an 1 -frame may 
contain 50(K>-50,(KI() bits of information, while a P-frame 
may contain UK)- 1(),0(MI bits of information. The P, and PB 
frames can Ix: iransinilted f;ister and reduce the congestion 
on the communic'tton link. 

The 11.245 and 'T. I20 proiociils and Poinl-to-Point Pro- 
tocol ("PPP") are also u.sed during audio/video conferenc- 
ing. 11.245, entitled "Control Protocol for Multimedia 
Comnninicalion," de lines initiation of comnnimcations 
belwecn systems and negotiation of capabilities prcjcedures. 

T.I 20, entitled "User Data Transmi.ssion Using a Multi- 
Layered Protocol ("MLP") is the TTU-'T recommendation 
for data conferencitig. PPP is u.sed to encx)de network layer 
data over a serial communication link. For more information 
on 1 1.320, 11.323, 1 1.324, 1 1.26 1, 1 1.263, 11.245 and T. 1 20 see 
"Mdinsfmnu Vuteocoiiferencini;: A Dercfopcr's dude to 
Distance Multimedia'' by Joe Duran and Charlie Sauer, 
Addi.st^»n, Wesley, Reading, Ma.ss., 1997 or Videoconfer- 
encing and Vtdeoieleplumy: Technofoi^y and Slinidards'\ by 
Richard Schaphorst, Artech llou.se, Norwood, Ma.ss., 1996. 
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For more informalion on PPP sec Internet Request For IND 16 translates the packets from the DVR queue into 

Comments ("Ri"C") 1060. ITie above documents are 11.324 protocol, and synchronizes the packets for output on 

expressly incorporated herein by reference. second communication link 20 to second computer network 

Audio, video, data and control information (G.7xx, 12 (e.g., provides audio/video "lip" synchronization). The 

H.26 1, H.263,H.245/r. 120) is encoded in 11.225 frames for 5 M.324 packets are output as 11.223 frames using the 

u.se with M.323 and H.223 frames for use with H.324. ITie sequence numbers and the appropriate timing as is illus- 

11.225 and M.223 frames are data link frames used in the OSI trated by timing marks Tl'- r6'. Packets PI, P2 and P3 are 

data link layer over first and second communication links output on liming marks Tl' 17' and 13' respectively. Packets 

(18^0). The video codec frames (J, P, and PB) are encoded P6 and P5 have been re -ordered in the proper sequence and 

in 11.225 or 11.223 frames. ITie multimedia information lo output on timing marks T5' and T6' respectively. Thus, 

encoded in the 11.225, H.223 or other frames is hereinafter M.324 packets arc output with the proper timing and packet 

referred to as "encoded-information-typcs." sequencing by IND 16. IND 16 gives H.320 and H.324 a 

On the larger bandwidth communication link 18 using constant stream of output packets even though the M.320 and 

H.323 (i.e., LAN or other network side) contention losses, Ii.324 do not include timing values with each packet. No 

retries and unreliable links indicate protocol information 15 audio/video synchronization or "lipsync" is required for 

packets may be lost or delayed an indeterminable amount of ll.323-to-M.323 translation (e.g., M.323/LAN-* H.323/ 

time. ITius, first computer network 12 may use Real-Time PPP). 

Protocol ("RTP") in its protocxil stack to provide a timing PIG. 4 is a block diagram 48 illustrating a DVR queue 50 

synchronization and sequence number mechanism when used in IND 16. DVR queue 50 illustrates the packet 

M.323 is used. K\V allows the H.323 to compensate tor 20 sequence received from first computer network 12 in FIG. 3. 

packets varied in time and lost packets. However, H.323 In a preferred embodiment of the present invention, DVR 

may use other protocols for packet timing and sequencing. queue 50 is a predetermined fixed length. If first communi- 

'Hie H.324 protocol typically does not provide any timing cation link 18 is a larger bandwidth than second conimuni- 
rclationships like those provided with RTP in H.323. i'hus, cation link 20, DVR queue 50 may ovcrllow and frames 
IND 16 provides packet synchronization after translation 25 containing the packets arc discarded as is illustrated in 
before sending the packet to the H.324 side. A DVR queue method 24. Software ser\'cr 22 in IND 16 uses DVR queue 
in IND 16 is used to provide packet synchronization and 50 to provide packet sequencing and packet timing synchro- 
sequencing. Packet and time sequencing is not required for nization when translating between two network protocols 
H.323-to-H.323 (e.g., H.323/LAN-*H.323/PPP) (e.g., H.323 and H.324) that require timing synchronization, 
translation, and DVR queue is used as a bulTer in IND 16 for 30 Method 28 (FiGvS. 2A and 2B) is illustrated with one 
such an embodiment. embodiment of the present invention. First computer net- 

MG. 3 is a block diagram 46 illustrating packet synchro- work 12 (FIG. 1) uses H.323 and sends H.225 frames with 

nization in IND 16. As is shown in FIG. 3, IND 16 receives multimedia informalion (e.g., audio, video and data 

H.323 packet information (e.g., audio, video and data information) as encoded- in format ion- types to IND 16 over 

inforinalion) from first computer network 12, over first 35 first coinnnuiication link 18 which has first bandwidth (e.g., 

communication link 18. First communication link 18 is a 10 Mb]>s). Second computer network 14 uses H.324 and 

larger bandwidth communication link (e.g., a 10 Mbps LAN receives H.223 frames with the encoded-information types 

communication link) than second communication link 20 from IND 16 over .second communication link 20 which has 

(e.g., a 56,000 bps serial line communication Unk). The a second smaller bandwidth (e.g., 28,800 bps) without undue 

H.323 packets are sent as H.225 frames over communication 40 delay or loss of information. However, the present invention 

link 18 to IND 16. IND 16 translates H.323 into H.324 and is not limited to this illustrative embodiment and other 

sends H.324 information to second computer network 14 embodiments may be used and the H.323 protocol can be 

over second communication link 20 as 11.223 frames. H.324 translated into H.320 and H.323 by IND 16. 

and H.320 provide timing and .sequencing relationships at a At step 30 (FIG. 2 A) of method 28, multiple 11.225 frames 

source. However, H.324 or H.320 do not provide any 45 arc received for the H.323 protocol over fust conn nutiicatioii 

in-.strcMm timing as is provided by RTP in H.323. H.324 and link 18. The ninlliplc iVnnies include encoded mullimcfli^i 

H.320 align data streams at a source and then "assume'Mhai informalion (e.g., encoded audio, video and data 

the underlying transmission medium will not cause an inlbrmation) as cncoded-informat ion-types. The H.225 

•'jitter" (i.e., timing variations in the transmitted frames). As frames include multiple video codec encoded-information- 

a result, IND 16 provides packet liming synchronization and S\) types shown in Table 1. The received 11.225 frames are 

packet sequencing in the DVR queue before sending H.324 stored in DVR queue 50 (FIG. 4) in a memory of IND 16 at 

data packets !o .second ccmiputer network 14. IND 16 step 32. The 11.225 frames with H.323 data from DVR queue 

removes tran.sinission "jiller" as well as provides "lip syn- 50 are translated into H.223 frames for H.324 at step 34. 

chronization" for II.323-i^<i-II.324 translation. For, 11.323/ DVR queue 50 is used to provide packet .sequencing and 

LAN-*<-H.323/PPP translation, no timing synchronization 55 packet liming synchronization. Ilie translated H.223 frames 

is required, s*} IND 16 does not provide either jitter removal are sen! over second communication link 20 al step 36. 

or lip synchronization, but does provide the data rale adap- Periodically (e.g., once }Kr second), a test is completed at 

tation described above (i.e., 10 Mpbs-t><-56,()00 bps or step 38 (1' IG. 2li) to determine whether the number of 11.225 

other nuxlem speeds). frames containing multimedia informalion arriving on (irst 

As is shown in FIG. 3, lirsl computer network 12 sends six (it) network connection 18 exceeds a predetermined queue 
H.323 protocol packets numbered PI, P2, P3, P4, P6, and P5 congestion threshold for DVR queue 50. 1'hc queue con- 
to IND 16 over first communication link 18. The packets geslion threshold is determined based on the size of DVR 
would l>c received in H.225 frames. Packet P6 was sent queue 50. If the queue congestion thre.sliold is exceeded, an 
before packet P5 (i.e. is out of sequence). Ilie packets are encoded-information-type is .selected at .step 40 (e.g., I i,263 
sent in a 'bursty" manner as is illu.strated by the timing fis 1-frames encoded in H.225 frames). 

marks nT-T6. IND L6 accepts the packets in H.225 frames For example, 11.263 1-frames are video codec Irantes 

and .stores them in the DVR queue. witk)Ul any motion compensation predictions, and H.263 
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P-lYamcs, frames, and PB-t'ramcs include errors tor motion 
compensation prcdicaiion. The H.263 and IMi frames can 
be discarded and only the H.263 I -frames encoded in H. 2 25 
frames are used to transmit audio/video conferencing data. 
I -frames are sent periodically and allow a video decoder to 
recover Irom any motion prediction errors generated by P 
and 1*13 frames. 

11.225 frames that do not have the selected cncodcd- 
information-iype (e.g., H.225 frames with encoded P, and 
PB frames) arc discarded at step 42 until the number of 
frames arriving on first communication link 18 is less than 
the predetermined queue congestion threshold. 11.225 
frames with the selected encoded- in formal ion-type (e.g., 
H.263 l-framcs) are stored in DVR cjueue 50 at step 32 to 
allow translation between H.323 and H.324 at step 34. When 
enough frames in the DVR queue are processed so the DVR 
queue reaches a predetermined length at step 38, discarding 
of frames that do not have the selected encoded-information- 
type is discontinued at step 44. 

Since one of the communication links typically has a 
larger communication bandwidth than the other communi- 
cation link, method 24 is used to compensate for the 
ditferences in communication bandwidth, timing and 
sequencing without undue delay or loss of information. 
Method 28 is dcscrilitd with respect to video codec infor- 
mation (i.e., 11.263 video code I, P, and PB frames). 
However, method 28 can also be used with audio informa- 
tion from the audio codecs described above (e.g., selecting 
between audio codec data for lower and higher fidelity 
audio) and with data information. Method 28 can also be 
used for translating protocols that do not require timing 
synchronization with a DVR queue. 

In an another embodiment of the pre.seni invention, 
X- number of frames which do not have the selected 
ericodcd-information-lype arc discarded at step 42 until 
Y-number of frames with the selected encoded-infomiation- 
type are stored in DVR queue 50 at step 32 before checking 
the queue congestion threshold again at step 38. 

In another embodiment of the present invention, a How 
control me.ssage is sent by IND 16 over iirst communication 
link 18 to first computer network 12 to practice frame 
discarding at step 42. In such an embodiment, a How control 
me.ssage is sent via the 11.245 protocol and requests first 
computer network 12 send frames with encoded- 
iiiforiiiatiun-types at a rate less than the available liandwidth 
on first coinniuin'cjilion link* 18 so more fraiiics are pro- 
ce.ssed. A timer is set on INI) 16 with i\ predetermined 
congestion control (e.g., I second) value to allow IND 16 to 
process the frames storetl in DVR queue 50. When the timer 
expires, a second control How message is sent with l i.245 
protocol lo inform Iirst computer network 12 that il may 
send frames with encoded-information-types again using the 
available bandwidth on lirsi communication link 18. 
Adaptive Prioriti/aticm of Multimedia Protocols 

In a preferred emlx:»diment of the present invention, 
selecieil frames are discarited at step 42 from DVR queue 50 
with method 52. PIG. 5 is a Mow diagram illustrating a 
method 52 for discarding received frames from DVR queue 
5((. At step 54, a first location in DVIi queue 50 is deter- 
mined containing a first frame with the .selected encoded- 
in form at ion -type (e.g., 11.225 frame with an encoded 11. 263 
1 -frame). At step 56, a second location is determined con- 
taining a second frame with the selected eiicoded- 
in formation-type. The frames (P, and multiple V\i frames) 
lietween the first and .second locations are discarded at step 
58, thereby increasing sj>ace in DVK (|uciie 5(1. As was 
discus.scd above, encoded 11.263 P, and PB frames can be 



50 



00 



discarded since encoded l-framcs alone can be used to 
display video information. If a first location in DVR queue 
50 containing a first frame with the selected encoded- 
in form at ion type cannot be found at step 54, frames are 
discarded from the DVR queue until a frame with the 
selected encoded-information type arrives. Method 52 can 
also be used to discard frames from a buffer used for 
translating protocols that do not require liming synchroni- 
zation with a DVR queue. 

RG. 6 is a block diagram 60 illustrating method 52. First 
location 62 (IIG. 6) determined at step 54 (PIG. 5) contains 
a first I -frame encoded in a 11.225 frame. Second location 64 
(PIG. 6) determined at step 56 (PIG. 5) contains a second 
encoded I -frame. The encoded P, and multiple PB frames 
between first and second locations (62,64) are discarded at 
step 58 thereby increasing space in DVR queue 50. 

In another embodiment of the present invention, selected 
frames are discarded at step 42 from DVR queue 50 with 
method 66. FIG. 7 is a flow diagram illustrating a method 66 
for discarding received frames from the DVR queue 50. At 
step 68, a first location containing a first frame with the 
selected encoded -in form a lion- type (e.g., H.225 frame with 
an encoded 1- frame) is determined. At step 70, frames in 
locations before first location 62 in the queue are di.scardcd 
thcrcliy increasing space in DVR queue 50. Method 66 can 
also be used lo discard frames from a buffer used lor 
translating protocols that do not require timing synchroni- 
zation with a DVR queue. 

FIG. 8 is a block diagram 72 illustrating method 66. At 
step 68 (FIG. 7), a first location 74 (FIG. 8) is determined 
containing a 11.225 frame with an encoded I-frame. At step 
70 (FIG. 7), the 1 1.225 frames with encoded P, and multiple 
PB frames before first location 74 (FIG. 8) arc di.scarded 
thereby increasing space in DVR queue 50. 

In another embodiment of the present invention, selected 
frames are di.scarded at step 42 as they are received with 
method 76 and are not stored in DVR queue 50. FIG. 9 is a 
How diagram illustrating method 76 lor di.scarding frames as 
they arc received. At .step 78, an average time period is 
determined when a frame with a .selected e n coded - 
in format ion -type is received (e.g., a 11.225 frame with an 
encoded H.263 I-trame). At step 80, a timer is set with the 
average time period. At .step 82, frames are discarded until 
the timer expires or a frame with ttie selected encoded- 
in format ion -type is received. Method 76 can also be u.scd lo 
discard frames iVt^m a buffer used fVir Ininslaling protocols 
that do not require timing synchronization with a DVR 
(|ueue. 

FIG. 10 is a block diagram 84 illustrating the method of 
I'lG. 9. It is determined a I step 78 that a H.225 frame with 
a selected encoded-information- type (e.g., an l-l'rame) is 
received every 0.5 sectuuis at locations 86, 88 and 90 
corresponding to time periods of zero, 0.5 and one second. 
A timer would be .set with the average time period of 0.5 
seconds step 80. Frames are di.scarded uniil the timer expires 
or a frame wilh the selected encoded- in form ;ition-ty[x; is 
received at step 82. The average time period determined is 
typically determined for a longer time period than is shown 
in FIG. 10 (e.g., over 1-2 minutes). The average time period 
may also be adjusted dynamically as the content of the 
audio/video conferencing information changes. 

In another embodiment of the present invention, a request 
may be made liy a computer network to select a new 
encoded-information-type for a specified time period (e.g., 
to send data) and discard all other frames with other 
eiouled-informat ion-types received during tlie S|x:cilied 
lime period. 
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FIG. 11 is a flow diagram illustrating method 92 for 
receiving Iramcs with a new encoded-information-typc. At 
step 94, a selection input is received from a computer 
network indicating a time period for continuously receiving 
frames with a new selected encoded -in form at ion -type. For 
example, IND 16 may receive a selection input from first 
computer network 12 for establishing a virtual data channel 
for T.120 data. T.120 could be used to send a spreadsheet or 
other collection of information data during the audio/video 
conference. At step 96, a timer is set with the indicated lime 
period received in the selection input. At step 98, other 
frames received with encoded information (e.g., H.263 I, P, 
and PB frames) are discarded that do not have the new 
selected cncoded-inforiiialioii-lype (e.g., T.120) until the 
timer expires. 

Adaptive Prioritization Systems 

FIG. 12A is a block diagram illustrating a protocol 
translation system 100 for one illustrative embodiment of 
the present invention. A first multimedia computer 102 is 
connected to first computer network 12, which is a LAN. 
First computer network 12 is connected to IND 16 with first 
communication link 18 (e.g., a LAN connection at 10 
Mbps). Multimedia computer 102 is using 11.323 protocol 
for audio/video conferencing with a second multimedia 
computer 104 using 11.324 protocol. Second multimedia 
computer 104 is connected to a second computer network 
14, which is a serial communications device (e.g., a 
modem). Second computer network 14 uses a second com- 
munication link 20 (e.g., a serial telephone line at .^6,000 
hps, an ISDN line, etc.) to connect to INO 16. 

INO 16 includes a LAN interface 106, a audio/video 
synchronization device 108 and a modem interface 110. 
Multimedia information (voice, video and data) is sent from 
multimedia computer 102 from first computer network 12 to 
IND 16 over cornmiinicalion link 18 as 11.225 frames. 
Communication link 18 may have multiple virtual channels 
for audio, voice and data information. LAN interface 106 in 
IND 16 sends 112 audio information via RTP and .sends 114 
video information via U TP to audio/video synchronization 
device 108. RTP was explained above. r. l 20dala Lssent 116 
from LAN intertxicc 106 to software server 22 using I'rans- 
mission Control Protocol (" FCP") or another data transmis- 
sion protocol known in the art. However, I'CP may or may 
not terminate on IND 16 depending on the type of data 
protocol being used. In addition, other protocols could al.su 
be used lo send audio, video or data information. 

Audio/video synchronization device 108 is used .synchrti- 
nize audio and video information .so audio information 
corresiXMids lo a video image. For example, lo.ss of lip 
synchroni/Jition I'or an audio/video conferencing call 
between two people is highly undesirable, so audio and 
v'dco inlormalton is synchronized. In one embodiment of 
ttie present invention, audio/video synchronization device 
108 uses DVk queue 50 (l-IG. 4) along with software 
comprising packet .sequencing and packet timing synchro- 
nization fnnctionnlity. DVk (jueue 50 is used to provide 
packet sequencing and packet liming .synchronization when 
translating between two network protocols as was explained 
above and illustrated in FIGS. 3 and 4. 

Software server 22 uses method 28 and one or more ol" 
methods 52, 66, 76 antl 92 described al>ove to translate 
11.323 proti»col infomiation into 1 1.324 protocol information 
and u.^es protocol translation tables 24 and 26 (I'lG. 1). 
Software server 22 receives synchronized audioA'ideo data 
over connection 118. Software server 22 ii.ses audio codec 
Protocol Data Units (^'PDUs ") such i\s PDUs for the G.723. t 
audio aidec known in the art to .send 1 20 syncbroni/ed audio 



i0,880 

14 

information to modem interface 110. However, olher audio 
codec PDUs could also be used (e.g., G.7 J1, G.722, G.728, 
or G.729). Software server 22 u.ses a video codec PDU such 
as a H.263 video codec PDU to send 122 synchronized video 

5 information lo modem interface 110. However, other video 
codec PDUs could also be used (e.g., 1126 1). Software 
server 22 sends 124 any TI20 data received on TCP input 
118 to modem interface 110 as data PDUs for a LAPM or 
other modem interface protocol known in the art. However, 

to other protocols could also be used (e.g., V. I4). 

Software ser\'er22 translates H.323 protocol information 
encoded in H.225 frames into H.223 frames with H.324 
protocol information using audio/video synchronization 
device 108 and data input 116 and sends the H.324 protocol 

15 information as H.223 frames to modem interface 110. 
Modem interface 110 outputs H.324 protocol to first com- 
puter network 14 (i.e., modem) which is connected to 
multimedia computer 104. Multimedia computer 104 can 
also initiate an audio/video conference with multimedia 

20 computer 102, and the audio/video conferencing communi- 
cation just described would be executed in reverse order 
(i.e., IL324-*H.323). 

FIG. I2li LS a block diagram illustrating a protocol 
translation system 125 for a preferred embodiment of the 

25 present invention. A first multimedia computer 102 is con- 
nected lo first computer network 12 that is a LAN. First 
computer network 12 is connected to IND 16 with first 
communication link 18 (e.g., a LAN connection at 10 
Mbps). Multimedia computer 102 is using H. 323/1, AN 

30 protocol for audio/video conferencing with a second multi- 
media computer 104 using 11.323/PPP protocol. Second 
multimedia computer 104 is connected to a second computer 
network 14, which is a .serial communications device (e.g., 
a modem). Second computer network 14 uses a second 

35 conunnnication link 20 (e.g., a serial telephone fine at 
56,000 bps, an ISDN line, etc.) to connect to IND 16. No 
audio/video synchronization is required to translate between 
H.323/LAN and H.323/PPP so audio/video synchronization 
device 108 and DVR queue SO are replaced by buffer 107. 

40 LAN interface 106 in IND 16 sends 112 audio information 
via RTP and sends video information 114 via RIT to butler 
107. T.120 data is optionally sent 116 from LAN interface 
106 to bulVer 107 or directly to software .server 22 with TCP 
or another transmission protocol. Audio/video stream and 

45 optional data stream 119 is sent from bulTcr 107 to .software 
server 22 for translation. Stdlware server 22 translates 
1 1.323/ LAN into H.323/PPP protocol infomiation using and 
sends the M.323/PPP protocol information to modem inter- 
face 110. Modem interface 110 outputs the 11.323/PPP 

50 protocol lo lirsi computer network 14 (i.e., modem) which is 
connected to mullimedia et>niputer 104. Multimedia com- 
puter 104 can also initiate an audio/video conference with 
mullimedia aompuler 102, and the audio/video conferencing 
communication just described would be executed in reveise 

55 order (i.e., II.323/PPP->H.323/LAN). 
Visual Congestion Inrlicalors 

In one emlx)diment of the present invention, IND 16 
includes a visual indication ol any congestion in DVR queue 
50 or a bull'er used in place if DVR queue 50. FIG. 13 is a 

60 block diagram illustrating a visual queue cxinge.stion indi- 
cator 126. Visual Queue congestion intlicalor 126 includes a 
first congestion indicator 128, a second congestion indicator 
130 and a Ihird congestion indicator 132 which are three 
difTerenl colors (e.g., green, yellow and red). Only one 

fis congestion indication 128, 130 or 132 is displayed at any 
instance of time. Vor example, in FIG. 13, (udy second 
congestion indicator 130 is displayed. Hie congestion indi- 
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cators are used as a visual indicalion of the amount of 
congestion in DVR queue 50 or buffer and can be imple- 
mented in hardware as Light Emitting Diodes or other 
hardware components on IND 16 or in software as graphical 
shapes on a display associated with IND 16. The visual 
congestion indicators (128,130,132) arc not limited to the 
circle shapes shown in FIG. 13. 

Tor example, the first congestion indicator 128 (e.g., a 
green color) may indicate no conge.stion or a small amount 
of congestion. In one embodiment of the present invention, 
when first congestion indicator is displayed, methods 52 and 
66 are used by IND 16 to discard frames from DVR queue 
50 or bufi'er thereby increasing space in the DVR queue or 
buftcr. When second congestion indicator 130 is displayed 
(e.g., a yellow color) an intermediate amount of congestion 
is occurring in DVR queue 50 or buller. IND 16 uses method 
78 to discard frames until a timer expires or a frame with a 
selected encoded-information-lypc is received. Tor example, 
all n.263 V, and Pli frames encoded in H.225 arc di.scarded 
until an M.263 I -frame is received. When third congestion 
indicator 132 is displayed (e.g., a red color), a major amount 
of congestion Ls occurring in DVR queue 50 or butfcr. IND 
16 .sends a M.245 control message to ask a computer network 
to .send frames at less than the available bandwidth for a 
determined period of time. IND 16 may afso drop alt fraines 
received until the frames in DVR queue 50 or butfcr are 
procc.s.sed. 

I'IGS. I4A, 14B and 14C are a ttow diagram illustrating 
a method 134 for displaying a visual indication of conges- 
tion in DVR queue 50 or bulYcr. At step 136 (FIG. 14A) a test 
is conducted to determine if a first level of congestion is 
occurring in DVR queue 50or bulTer If so, at step 138, first 
visual indicator 128 is di.splayed (e.g., with a green color). 
An cncoded-informaiion-type encoded in a data frame is 
selected <nl step 140 of FIG. I4A (e.g., 11.263 I-frarne 
encoded in a M.22.5 frame). Ail frames in DVR queue 50 or 
bulfer that do not have the selected encoded-information- 
type arc discarded at .step 142. 

If the first level of conge.stion is not occurring at .step 136, 
at .second test is conducted at step 144 (FIG. 14B) to 
determine if a second level of congestion is occurring. If .so, 
at step 146 second visual indicator is displayed (e.g., with a 
yellow color). An encoded- in format ion-type encoded in a 
frame is .selected at step 148. All additional frames received 
arc discarded that do not have the selected encodcd- 
infornial ion-type at step 150 (e.g., all 11.263 \\ and PR 
frames), l-rames that do have the .selected encoded- 
information-iype are added to DVR queue 50 or bulfer The 
frames already in DVR queue 50 or bulfer are processed as 
now received frames are being discarded. 

If the second level of congestion is not occurring at step 
144, a third test is conducted at step 152 (l-Kr I4C) to 
determine if a third level of congestion is occurring. If so, at 
step 154 third visual indicator 132 is di.splayed (e.g., with a 
red c(^l(»r). An encoded-information-type encoded in a frame 
is selected at step 156. All frames in DVR are di.scarded in 
DVR tjueue 50 or bulfer at step 158. All additional frames 
received are di.scarded at step 158 until a frame with the 
selected encoded-information-type is received. 

If Ihc third level of congestion is not occurring at .step 152, 
then no congestion indicators are displayed at step 160 
indicating no congestion in DVR queue 50 or bulfer. Method 
134 and vi.sual queue congestion indicator 136 are u.sed in 
IND 16 to allow a user to visually delermine the amount of 
congestion in DVR queue 50 or a bulfer. Method 134 and 6S 
visual (|Mene congestion indicator 136 arc beneficial to 
indicate congestion in DVR queue 50 or bulfer when a first 
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communication link connected to IND 16 has very large 
bandwidth and a second communication link has a smaller 
bandwidth (e.g., 10 Mbps LAN communication link and a 
28,800 bps serial communication link). Method 134 and 
5 visual queue congestion indicator 136 can also be used with 
other queues or buffers that are not implemented as DVR 
queue 50. 

In one specific embodiment of the present invention, IND 
16 (FIG. 12) is implemented in a Total Control Enterprise 
Network Hub commercially available from 3Com Corpora- 
tion of Santa Clara, Calif. The Total Control product 
includes multiple network interface cards connected by a 
common bus. See "Modem Input/Output l^ocessing Signal- 
ing Techniques", U.S. Pat. No. 5,528,595, granted to Dale 
M. Walsh et al. for a description of the architecture of the 
Total Control product, which is incorporated by reference 
herein. For example, LAN interface 106 is a 3Com lithcrnet 
or Token Ring interface card other Ij\N interface card. 
Modem interface 110 is a 3Com Quad Modem card (or the 
equivalent). Computer software server 22 is added to exist- 
20 ing software in the Total Control product (such as in the 
Ldge.server card) to implement method 28 and one or more 
of methods 52, 66, 76, 92 and 134 described aliovc to 
accomplish features described herein. However, IND 16 can 
also be implemented in other devices with other hardware 
2^ and software configurations and is of course not limited to 
iinplenientation in a Total Control product or the equivalent. 

It should be understood that the programs, processes, 
methods and apparatus described herein are not related or 
limited to any particular type of computer apparatus 
(hardware or .software), unless indicated otherwise. Various 
' types of general purpo.se or specialized computer apparatus 
may be u.sed with or perform operations in accordance with 
the teachings described herein. 

In view of the wide variety of embodiments to which the 
principles of the invention can be applied, it should Itc 
understood that the illustrated embodiments are exemplary 
only, and should not be taken as limiting the scope of the 
present invention. For example, the steps of the tlow dia- 
grams may be taken in sequences other than those described, 
and more or fewer elements may be used in the block 
40 diagram. s. 

The claims should not be read as limited to the described 
order or elements unless stated to that effect. In addition, u.se 
of the term "means" in any claim is intended to invoke 35 
U.S.C. §112, paragraph 6, and any claim without the word 
*■ means" is not .so intended, fherefore, all embodiments thai 
come within the scope and spirit of the following claims and 
equivalents thereto are claimed as the invention. 
Wc claim: 

L A mellnid of adapt ively prioritizing between two or 
more encoded-informat ion -types received over a communi- 
cation link in a plurality of frames in an internetworking 
device, the frames including a plurality of data liil.s, I he 
incihod comprising the following .steps: 

receiving a plurality of frames for a first network proU>col 
over a fir.st communication link having a first commu- 
nication bandwidth, the plurality of frames having one 
of a plurality of encoded-infonnation-types; 
storing the received frames in a delay variance iximoving 
queue in a memory in the internetworking device, the 
delay variance removing queue allowing the inlcrnet- 
working device to compensate for.sequencing or liming 
relationships used in the first network protocol; 
translating the frames from the delay variance lentoviiig 

queue into a .second network protocol; 
determining (Kiriodically whether a number of frames 
arrivitig on the (irst nelwork connection exceeds a 
predetermined queue congestion threshold, and if .st>. 
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selecting an cncoded-information-lypc encoded in a 
frame to store in the delay variance removing queue; 

discarding received frames that do not have the selected 
encoded-informalion-type until enough frames in the 
delay variance removing queue have been processed so ^ 
the delay variance removing queue reaches a predeter- 
mined length; and 

sending (he translated frames for the second network 
protocol over a second communication link having a 
second communication bandwidth, the second commu- 
nication bandwidth being less than the lirsl communi- 
cation bandwidth. 

2. A computer readable medium having stored therein 
instructions for causing a central processing unit to execute 
the method of claim 1. 

3. 'Hie method of claim 1 wherein the step of discarding 
receivcfi frames includes: 

determining a first location in the delay variance remov- 
ing queue containing a first frame with the selected 
c ncoded - in for ma 1 io n - 1 ype ; 

determining a second location in the delay variance 
removing queue containing a secx)nd frame with the 
selected encoded-information-typc; and 

discarding other tramcs in other locations in the delay 
variance removing queue between the iirst and second 
locations, thereby increasing space in the delay vari- 
ance removing queue. 

4. The method of claim I wherein the step of discarding 
received frames includes: 

determining a first location in the delay variance remov- 
ing queue containing a Iirst frame with the selected 
encoded-informalion-type; and 

discarding other frames in locations in the delay variance 
renioviiig queue prior to the first location, thereby 
increasing space in the delay variance removing queue. 

5. Hie method of claim 1 wherein the step of discarding 
received frames includes: 

discarding frames that do not have the selected encoded- 40 
information-type until X-number of frames have been 
received or a frame with the selected encoded- 
informalion-type is received. 

6. 'Hie method of claim 1 wherein the step of discarding 
received frames includes: 45 

sending a first How control message over the first com- 
munication link requesting frames be sent at less than 
ihe fiiMt communication bandwidth; 

setting a timer with a predetermined congestion control 
value; and 

upon expiration of the timer, sending a second (low 
c{-.:ilrol message over the first comnuinicalion link 
rcquesling frames again be sent at the first communi- 
cation l.iandwidth. 

7. The method of claim 6 wlierein the first and secimd 
How control messages are sent with the 11.245 control 
protocol for multimedia comniunicalitMi. 

8. fhe method of claim I wherein the step of discarding 
received frames inchides: "''^ 

determining a time period when a Ira me with the selected 

encoded-informalion-type is received; 
setting a timer with the time period; and 
discarding frames received until ilie timer expires or a 65 

frame with Ihe selected encocled-informatioti-tyiK: is 

received. 



9. The method of claim I further comprising: 
receiving a selection input with a time period for con- 
tinuously receiving frames with a new encodcd- 
information-type; 

setting a timer with the time period; and 
discarding frames received that do not have the new 
encoded- in formation -type until the time expires. 

10. 'Hie method of claim I wherein the first commiinica- 
tion link is communication link to a local area network 
communication link and the second communication link is a 
serial line communication link. 

11. 'Ilic method of clam 1 wherein the first network 
protocol is 11.323 and the second network protocol is any of 
11.320, n.323 over a LAN, M.323 over PPP or 1 L324. 

12. 'fhe method of claim 1 wherein the step of selecting 
an cncoded-in formation-type includes .selecting a 11.263 
I-frame as the encoded-in formation-type. 

13. 'fhe method of claim 1 wherein the discarding step 
includes discarding received frames having an encoded- 
information-type of any of H.263 P-frame.s, frames or 
PIJ- frames. 

14. The method of claim I wherein Ihe step of discarding 
received frames includes: 

determining whether a first frame with the selected 
encoded- in form at ion-type is in the delay variance 
removing queue, and if not, 

discarding received frames that do not have the selected 
encoded-in format ion- type until a frame with the 
.selected encoded -in form at ion -type is received. 

15. fhe method of claim I wherein Ihe first and second 
communication links have two or more virtual channels for 
sending and receiving frames and the two or more virtual 
channels include a first virtual channel I'or audio 
information, a .second virtual channel for video information 
and a third channel for data information. 

16. Hie method of claim 1 wherein Ihe plurality of f rames 
received over the communication link include any of 11.225 
or 11.223 frames. 

17. 1lie method of claim I wherein the delay variance 
removing queue is a buflcr that is used to translate network 
protocols that do not require timing synchronization with a 
delay variance removing queue. 

18. An internetworking apparatus for translating a plural- 
ity of frames received for a Or.st network protocol over a first 
communication link having a first communication 
bandwidth, llie jihiralily of frames having one (tf a plural ily 
of encoded-information-types into a second nelwoi k proto- 
col sent over a second communication link having a .second 
comnuinicalion bandwidth, the apparatus comprising: 

a delay variance removing queue, f"or allowing the inler- 
nei working device lo compensate for sequencing or 
liming relationships used in a first network protocol; 

a translator, for translating frames from ihe nrst network 
protocol stored in the delay variance removing queue 
into a second network protocol; and 

a queue congestion threshold for discarding received 
frames that do have a selected encoded-informat ion- 
type unlil enough frames in the delay variance remov- 
ing queue are processed so the delay variance iximoving 
queue reaches a predetermined length. 

19. fhe internetworking apparatus of claim 18 wherein 
Ihe delay variance removing queue is a buffer that is used to 
translate network protocols that do not require timing syn- 
chronization with a delay variance removing queue. 
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